diff --git a/.gitattributes b/.gitattributes
index a6344aac8c09253b3b630fb776ae94478aa0275b..08fe72c2f1d234849f60ad174babd0ac762ae019 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -33,3 +33,13 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+presentation/reports/eval_finetune_curves.png filter=lfs diff=lfs merge=lfs -text
+presentation/reports/eval_finetune_curves_test3.png filter=lfs diff=lfs merge=lfs -text
+presentation/reports/finetune_curves.png filter=lfs diff=lfs merge=lfs -text
+presentation/reports/finetune_curves_ft-12L.png filter=lfs diff=lfs merge=lfs -text
+presentation/reports/finetune_curves_ft-ce.png filter=lfs diff=lfs merge=lfs -text
+presentation/reports/finetune_curves_ft-focal.png filter=lfs diff=lfs merge=lfs -text
+presentation/reports/finetune_curves_ft-scratch.png filter=lfs diff=lfs merge=lfs -text
+presentation/reports/finetune_curves_ft-synthetic.png filter=lfs diff=lfs merge=lfs -text
+presentation/reports/finetune_curves_ft-weighted.png filter=lfs diff=lfs merge=lfs -text
+presentation/reports/pretrain_curves.png filter=lfs diff=lfs merge=lfs -text
diff --git a/presentation/README.md b/presentation/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..86f4e9b597728da749e5038a21df6a8db97c5eb8
--- /dev/null
+++ b/presentation/README.md
@@ -0,0 +1,661 @@
+# BITS Phishing Detector
+
+Final project for the Skillspire AI & Machine Learning course (Nathan King, Build IT Systems LLC), presented
+late Oct / early Nov 2026. This README is the hand-off document for the instructor and for future BITS work.
+The running record of every decision, number and problem is `RUN_LOG.md` (entries 0-8 as of this
+writing); this file summarises it and explains how to reproduce it. Status as of 2026-10-08 14:00: data,
+tokenizer, smoke test and all `src/`, `scripts/` and `app/` code are done; MLM pretraining is running; the
+presentation notebook is in progress; fine-tuning, evaluation, export and deployment have not run yet.
+
+---
+
+## 1. What this is
+
+| | |
+|---|---|
+| Task | **Classifier** (not a regressor). Input: one email as `subject + "\n\n" + body` (plain text; sender, headers and attachments are never read). Output: probabilities for three classes, **Legitimate / Phishing / Spam**, plus the roll-up **Unwanted (phishing or spam)** = P(Phishing) + P(Spam). |
+| Use | Defensive triage: flag unwanted mail for a human to review. Not a filter that deletes mail, not a tool for writing lures. |
+| Model | One BERT-style pre-LayerNorm encoder written from scratch in plain PyTorch (`src/model.py`, no `transformers`, no pretrained weights): 24 layers, d_model 1024, 16 heads, FFN 4096, vocab 32,768 (own byte-level BPE), 512 tokens. **337,476,608 parameters** with the tied MLM head, **337,443,842** as the classifier. |
+| Training | Phase 1: masked-language-model pretraining on deduplicated Enron email. Phase 2: fine-tuning with a 3-way head. Rows whose corpus never separated spam from phishing (TREC-05/07, CEAS-08 positives) train on P(phishing)+P(spam) with a partial-label loss. |
+| Core challenge | Public corpora label "unwanted" mail, not "phishing vs spam"; they are 1998-2008 heavy, overlap each other and the pretraining corpus, and legitimate mail is about twice as long as spam. The work is mostly in honest data: deduplication, leakage checks and a test set nothing upstream has seen. |
+| Scope | **One strong model.** The experiments vary the instructor's dials (layers, epochs, loss, frozen layers, synthetic data, scratch vs pretrained init) on this one architecture; there is no architecture comparison. |
+
+## 2. Results
+
+### 2.1 Headline metrics (deployed run `ft-focal`, evaluated 2026-10-09 10:46)
+
+Model selection used the **validation** set only: `ft-focal` had the best validation F1 of the unwanted
+roll-up (0.9951) and the best validation three-way macro-F1 (0.9869) of the six runs in section 9. The two
+holdouts below were then evaluated once per run for reporting; `ft-focal` also leads on both of them.
+
+| TREC-06 holdout (16,302 emails, 23.9% unwanted; whole corpus unseen), threshold 0.5 | value |
+|---|---|
+| Recall, unwanted (reported first: a missed phish is the costly error) | **0.9741** (101 of 3,894 missed) |
+| Precision, unwanted | 0.8794 (520 false alarms on 12,408 legitimate; FPR 4.2%) |
+| F1, unwanted | **0.9243** (best achievable at any threshold: 0.9397 at p >= 0.68) |
+| Accuracy | 0.9619 |
+| ROC-AUC / PR-AUC | **0.9946** / 0.9848 |
+| Recall at 1% FPR / at 0.1% FPR | 0.8955 / 0.6497 |
+| Credential-lure slice (398 rows matching 37 lure phrases) recall / precision | 0.9959 / 0.9758 |
+
+TREC-06 is the hard, honest number: a different corpus (2006 TREC spam track) with its own labelling
+conventions. `reports/error_analysis.md` shows that many "false positives" are spam-looking mail that TREC-06
+labels ham (quoted spam inside mailing-list threads, "remove me from your list" replies, sweepstakes mail) and
+that the largest block of misses are "ScanMail virus found" notices TREC-06 labels spam. Legitimate TREC-06 mail
+is also ~3x longer than its spam and 24% of it is truncated at 512 tokens.
+
+| test_holdout3 (3,220 emails: 1,999 legit / 521 phishing / 700 spam; tokenizer never saw them) | Legitimate | Phishing | Spam |
+|---|---|---|---|
+| Precision / Recall / F1 | 0.9900 / 0.9935 / 0.9918 | 0.9961 / 0.9866 / 0.9913 | 0.9785 / 0.9757 / 0.9771 |
+| Macro-F1 (3 classes) / accuracy | **0.9867** / 0.9885 | | |
+| Binary roll-up (unwanted vs legitimate): F1 / recall / precision / ROC-AUC / PR-AUC | 0.9864 / 0.9836 / 0.9893 / 0.9993 / 0.9989 | | |
+| Phishing vs spam, given the mail is unwanted (1,221 rows) | 0.9959 (4 phishing called spam, 1 spam called phishing) | | |
+
+3x3 confusion matrix (rows true Legitimate / Phishing / Spam, columns predicted): `[1986, 1, 12]`, `[4, 514, 3]`, `[16, 1, 683]`.
+Three-way accuracy per source: Assassin 95.3% (n=236), CEAS-08 99.0% (n=736), Enron 98.4% (n=628), Nazario 98.6% (n=222), Nigerian_Fraud 98.7% (n=299), TREC-07 99.8% (n=1099).
+
+| Prediction time, one email, batch 1 (`reports/latency.json`, 100 TREC-06 samples, mean 280 tokens) | mean | p50 | p95 | vs 74 ms benchmark |
+|---|---|---|---|---|
+| GPU, bf16 (RTX 3090 Ti) | **10.1 ms** | 8.9 ms | 13.7 ms | 0.14x (faster) |
+| CPU, fp32, 16 threads | **459 ms** | 445 ms | 701 ms | 6.2x (slower) |
+
+Plots next to these files: `reports/confusion_matrix.png`, `confusion_matrix_test3.png`,
+`confusion_matrix3_test3.png` (3x3), `roc_curve*.png`, `pr_curve*.png`, `finetune_curves.png`,
+`pretrain_curves.png`, `eval_finetune_curves*.png`, `eval_pretrain_curves*.png`; written error analyses
+`error_analysis.md` / `error_analysis_test3.md`. The fine-tune experiment table is `finetune.py list` (section 9).
+Every run's own evaluation lives in `reports/eval_<run>/`.
+
+### 2.2 Measured facts that exist today
+
+| Fact | Value | Source |
+|---|---|---|
+| Parameters | 337,476,608 (MLM, tied head) / 337,443,842 (classifier) | `reports/smoke_test.json` |
+| Pretraining VRAM, micro-batch 16, no checkpointing | 13.65 GiB reserved, 23,642 tok/s | `reports/smoke_test.json` |
+| Pretraining VRAM, micro-batch 24, no checkpointing (**used**) | 17.65 GiB reserved, 19.80 GiB total device incl. desktop, 25,227 tok/s | `reports/smoke_test.json` |
+| Pretraining VRAM, micro-batch 24, gradient checkpointing | 8.06 GiB reserved, 19,156 tok/s | `reports/smoke_test.json` |
+| Measured bf16 matmul | 63.9 TFLOPS (4096^3); the smoke test's 25k tok/s is ~63% MFU of the 80 TFLOPS peak | RUN_LOG 0, 3 |
+| Convergence check (fresh init, 30 steps) | MLM loss 10.58 -> 7.67 (ln V = 10.40) | RUN_LOG 3 |
+| Latency, one 512-token email, GPU bf16 (RTX 3090 Ti), untrained-weight smoke test | 10.7 ms (trained checkpoint, real emails: section 2.1) | `reports/smoke_test.json` |
+| Latency, one 512-token email, CPU fp32, 16 threads, untrained-weight smoke test | 699 ms (319 ms at 128 tokens) | `reports/smoke_test.json` |
+| Pretraining plan `large-24L` | 12 epochs x 796 steps = 9,552 optimizer steps, micro-batch 24 x 11 = effective batch 264, peak LR 1.5e-4, warmup 573, cosine to 10% | `checkpoints/pretrain/large-24L/run_summary.json` |
+| **Pretraining result `large-24L`** (complete 2026-10-09 03:31) | 9,552 steps, 1.29B tokens, 15.05 h of training, 23,889 tok/s mean, peak 17.94 GiB reserved; train MLM loss 1.524, **val MLM loss 1.986, masked-token accuracy 62.9%** (val loss fell monotonically, no restart needed) | `checkpoints/pretrain/large-24L/run_summary.json`, `metrics.csv` |
+
+The smoke-test latencies were measured on untrained weights (latency does not depend on the weights); the
+numbers to quote are the trained-checkpoint measurements in section 2.1 (`reports/latency.json`). The
+instructor's 0.074 s/sample benchmark is met on GPU by a factor of ~7 and missed on CPU by a factor of ~6,
+which is why the Space runs on ZeroGPU.
+
+### 2.3 Data splits
+
+Binary build (RUN_LOG 1, `data_prep.py build`):
+
+| split | rows | positives | base rate | sources |
+|---|---|---|---|---|
+| finetune_train | 157,113 | 75,675 | 48.2% | TREC-05 46,386 / TREC-07 47,233 / CEAS-08 34,178 / Enron 24,223 / Assassin 5,093 |
+| finetune_val | 17,643 | 8,642 | 49.0% | same five, grouped by near-duplicate cluster |
+| test_holdout (TREC-06) | 16,302 | 3,894 | 23.9% | never seen by tokenizer, pretraining or fine-tuning |
+| pretrain (MLM) | 226,273 docs | - | - | corbt/enron-emails after dedup + quote stripping; 407.9M chars, 107.7M tokens |
+
+Three-way build (RUN_LOG 7, `data_prep.py build3`; label3 0 legit / 1 phishing / 2 spam / -1 unwanted-unsplit):
+
+| split | rows | legit | phishing | spam | unwanted (TREC/CEAS positives) |
+|---|---|---|---|---|---|
+| finetune_train | 160,263 | 81,438 | 3,150 | 12,870 | 62,805 |
+| finetune_val | 15,284 | 7,002 | 340 | 763 | 7,179 |
+| test_holdout (TREC-06, rows unchanged) | 16,302 | 12,408 | - | - | 3,894 |
+| **test_holdout3** | 3,220 | 1,999 | 521 | 700 | - |
+
+Plus `data/processed/synthetic.parquet`: 2,000 template-generated rows (1,000 legitimate, 1,000 phishing
+lures, 0 spam), training-only, off by default. finetune_val shrank from 17,643 to 15,284 rows when
+test_holdout3 was carved out, so val numbers from binary-era runs are not strictly comparable.
+
+---
+
+## 3. Setup on Fedora 44
+
+Machine used: Fedora 44 KDE, RTX 3090 Ti 24 GB (sm_86), NVIDIA driver 615.71.09 (CUDA 13.4), Ryzen 9 5950X,
+40 GB RAM, system Python 3.14.7, uv 0.12.5 (`~/.local/bin/uv`), tmux.
+
+```bash
+cd "/home/nathank/BITS/dev/Phishing Model"
+uv venv --python 3.14.7 .venv
+uv pip install --python .venv/bin/python --torch-backend=cu130 -r requirements-train.txt
+hf auth login                       # .venv/bin/hf; needed for the upload step only
+```
+
+Why these pins:
+
+- **`--torch-backend=cu130`, never `auto`.** The plain PyPI torch 2.14.1 Linux wheel is the CUDA 13.0 build;
+  driver 615 runs it by backward compatibility. uv 0.12.5's `auto` would pick cu132 on this driver. One
+  resolution installs everything (180 packages, all binary wheels, venv ~6.1 GB).
+- **Python 3.14.7, not 3.15.** torch.compile raises on 3.15, triton 3.8.0 requires <3.15 and `spaces` requires
+  <3.15. Always pass `--python 3.14.7`.
+- **`huggingface_hub==1.33.0`** because tokenizers 0.23.2 requires `huggingface-hub<2.0`.
+- **`fsspec==2026.7.0`** because datasets 5.1.0 caps `fsspec[http]<=2026.7.0` (2026.9.0 is newer and breaks the
+  resolve). Do not `uv pip install -U fsspec`.
+
+Verify the GPU (do not run while a training job holds the GPU unless you accept a small extra CUDA context):
+
+```bash
+nvidia-smi
+.venv/bin/python -c "import torch; print(torch.__version__, torch.version.cuda, torch.cuda.get_device_name(0), torch.cuda.is_bf16_supported())"
+# expected (2026-10-07): 2.14.1+cu130 13.0 NVIDIA GeForce RTX 3090 Ti True
+```
+
+---
+
+## 4. Project layout
+
+```
+Phishing Model/
+├── README.md                     this file
+├── RUN_LOG.md                    dated log of every run, decision, number, bug and fix (the "challenges" source)
+├── config.yaml                   every dial in one file (section 6)
+├── requirements-train.txt        pinned local training stack (Python 3.14.7, torch 2.14.1 cu130)
+├── .gitignore                    excludes .venv, data, checkpoints, logs, weights, tokens
+├── src/
+│   ├── common.py                 config loading, seeding, logging, text normalisation, dedup keys (shared by all)
+│   ├── data_prep.py              download -> dedupe -> decontaminate -> split -> leakage asserts (build/check/build3/check3)
+│   ├── tokenizer.py              byte-level BPE (NFKC, case kept): train on pretrain + finetune_train only; stats
+│   ├── model.py                  from-scratch pre-LN encoder, MLM head, classification head (copied to app/)
+│   ├── smoke_test.py             VRAM micro-batch finder, throughput, SDPA backend check, convergence, latency
+│   ├── pretrain.py               resumable bf16 MLM pretraining (train/export/plot)
+│   ├── make_synthetic.py         2,000 template-generated training rows (build/show)
+│   ├── finetune.py               3-way classifier fine-tuning with the instructor's dials (train/plot/list)
+│   ├── evaluate.py               the only reader of the holdouts: metrics, latency, plots, error analysis
+│   └── export.py                 bf16 safetensors + config + model card -> model_export/<run>/ and app/
+├── scripts/
+│   ├── launch_pretrain.sh        start or resume pretraining in tmux session `phish-pretrain`
+│   ├── run_experiments.sh        fine-tune queue: waits for large-24L/final, runs the six dial experiments,
+│   │                             evaluates each into reports/eval_<run>/, exports ft-ce to model_export/ft-ce and app/
+│   └── upload_to_hf.py           publish model repo + Space; dry run by default, `--yes` to write
+├── app/                          the Hugging Face Space
+│   ├── app.py                    Gradio UI: 3-class label, prediction time, token saliency, rule-based cues
+│   ├── README.md                 Space front matter (sdk gradio 6.29.1, python 3.12.12) + description
+│   ├── requirements.txt          Space pins (torch 2.13.0, spaces 0.51.3, ...)
+│   ├── .gitattributes            LFS rules for weights and tokenizer
+│   └── model.py, model.safetensors, config.json, tokenizer.json   written by export.py (not yet present)
+├── FinalProject_PhishingDetector.ipynb   presentation notebook in the instructor's 9-step pattern (section 5.10);
+│                                 written by the notebook step, in progress as of 2026-10-08 14:00
+├── reference/
+│   ├── FashionMNIST_Gradio_Full.ipynb   instructor's example notebook (the 9-step pattern)
+│   └── INSTRUCTOR_REQUIREMENTS.md       grading rubric, verbatim
+├── docs/research/
+│   ├── research_dataset-schemas.md      schemas, provenance and licences of every candidate dataset
+│   ├── verify_data.md                   independent verification of the data plan and holdout choice
+│   ├── research_model-math.md           parameter / memory / FLOP / latency math for the encoder
+│   ├── model_math.py                    pure-Python sizing calculator behind that document
+│   ├── research_torch-python.md         torch 2.14.1 / CUDA 13.0 / Python 3.14 compatibility, SDPA backends
+│   ├── research_other-libs.md           every other pin and its conflicts (fsspec, huggingface_hub, spaces)
+│   ├── verify_pins.md                   independent verification of the pins
+│   ├── research_gradio-spaces-hub.md    Gradio 6, ZeroGPU, Space front matter, Hub upload API
+│   └── bigshp_research.md               why the 300k "spam/ham/phish" set was rejected
+├── data/                         (git-ignored)
+│   ├── raw/                      HF cache for corbt, Champa mirror CSVs (aura/), inspected-only 300k set (bigshp/)
+│   └── processed/
+│       ├── finetune_train.parquet, finetune_val.parquet    training / model-selection splits (with label3)
+│       ├── test_holdout.parquet                            TREC-06 binary test set
+│       ├── test_holdout3.parquet                           clean-provenance three-way test set
+│       ├── pretrain.parquet                                MLM corpus (226,273 docs)
+│       ├── pretrain_tokens.npy/.json, pretrain_val_tokens.npy   cached token stream used by pretrain.py
+│       ├── synthetic.parquet, synthetic_report.json        template supplement + shortcut audit
+│       ├── tokenizer.json, tokenizer_report.json           BPE vocabulary + per-split length stats
+│       ├── splits_manifest.json, data_report.md            counts, removals, every leakage assertion
+│       └── binary_snapshot/                                binary-era splits that build3 starts from
+├── checkpoints/                  (git-ignored)
+│   ├── pretrain/large-24L/       live run: step-NNNNNNN/, metrics.csv, run_summary.json, final/ when done
+│   ├── pretrain/dev/             tiny self-test run (delete after Nathan's OK)
+│   └── finetune/<run>/           best/, last/, metrics.csv, epochs.csv per fine-tune run (dev, dev-focal = self-tests)
+├── reports/                      smoke_test.json; experiments.jsonl + finetune_curves.png + pretrain_curves.png are
+│                                 dev-only until the real runs regenerate them (section 9); metrics*, latency*,
+│                                 confusion/ROC/PR plots, error analyses, eval_<run>/ later
+├── model_export/<run>/           export output (created by export.py)
+└── logs/                         one timestamped log per script invocation + tmux stdout of the pretraining run
+```
+
+---
+
+## 5. How to run each phase, in order
+
+All commands run from the project root with `.venv/bin/python` (written `PY` below:
+`PY=.venv/bin/python`). Every script takes `--config config.yaml` before the subcommand and writes a log to
+`logs/<script>-<timestamp>.log`. Flags below were checked against each script's `--help` on 2026-10-08.
+**While pretraining holds the GPU, prefix any other Python command with `CUDA_VISIBLE_DEVICES=""` and do not
+start smoke_test, finetune, evaluate on GPU, or a second training job.**
+
+### 5.1 Data (`src/data_prep.py`)
+
+```bash
+$PY src/data_prep.py inspect        # optional: schema + 2 rows per dataset, streaming
+$PY src/data_prep.py build          # binary splits + pretrain corpus + manifest + data_report.md (~2.5 min, 6.9 GB RSS)
+$PY src/data_prep.py check          # recompute every binary leakage assertion from the written files
+$PY src/data_prep.py build3         # three-way labels, Champa phishing corpora, test_holdout3 (~1 min)
+$PY src/data_prep.py check3         # recompute binary + three-way assertions
+```
+
+Flags: `build --limit N` / `build3 --limit N` are dev-only subsets (`build3 --limit` writes to
+`data/processed/dev_threeway_limitN/`); `build --spam-policy {positive_class,map_to_zero,exclude}` (default
+`positive_class`); `build --no-strip-quotes`; `--shingle-cap N`; `build3 --bigshp {on,off}` overrides the
+rejected 300k set (leave off); `check3 --out-dir`.
+**Do not re-run `build` or `tokenizer.py train` while `large-24L` is running**: a new `pretrain.parquet` or
+`tokenizer.json` rebuilds the token cache and the run then refuses to resume. `build` also rewrites the
+binary splits, so `build3` must follow it.
+
+### 5.2 Tokenizer (`src/tokenizer.py`)
+
+```bash
+$PY src/tokenizer.py train          # byte-level BPE, vocab 32,768, NFKC, case kept; ~30 s
+$PY src/tokenizer.py stats          # length / truncation / [UNK] table per split -> tokenizer_report.json
+```
+
+`train --vocab-size N`, `train --limit-docs N` (dev). Trains only on `pretrain.parquet` + `finetune_train.parquet`;
+val and test files are refused by name and checked with an exact-hash tripwire.
+
+### 5.3 Smoke test (`src/smoke_test.py`) — GPU must be free
+
+```bash
+$PY src/smoke_test.py               # -> reports/smoke_test.json + printed summary
+```
+
+Useful flags: `--candidates 16,24,12,8`, `--ckpt-candidates`, `--steps 30`, `--skip-cpu`, `--n-docs 3000`,
+`--latency-samples 100`, `--cpu-threads 16`. It probes micro-batches with the real gradient-accumulation
+pattern, measures tokens/s, checks which SDPA kernel runs, runs a 30-step convergence check and times
+single-email inference on GPU and CPU.
+
+### 5.4 MLM pretraining (`scripts/launch_pretrain.sh` -> `src/pretrain.py`)
+
+```bash
+EPOCHS=12 MICRO_BATCH=24 scripts/launch_pretrain.sh     # how large-24L was launched (2026-10-08 12:28)
+tmux attach -t phish-pretrain                           # watch; Ctrl-b d detaches; Ctrl-C inside = save + clean exit
+RESUME=1 MICRO_BATCH=24 scripts/launch_pretrain.sh      # resume from the latest checkpoint
+```
+
+Environment variables: `RUN_NAME` (default `large-24L`), `EPOCHS` (script default 6; fixes the schedule
+length at first launch, ignored on resume), `MICRO_BATCH` (**script default 16; always pass 24 for this run**:
+16 gives effective batch 256 instead of the run's 264 and the resume is refused), `SESSION` (default
+`phish-pretrain`), `RESUME=1`. The script refuses to start if the tmux session already exists; after the
+trainer exits the pane waits for Enter (or `tmux kill-session -t phish-pretrain`). Stdout is teed to
+`logs/pretrain_<run>_<timestamp>.out`.
+
+Direct use: `$PY src/pretrain.py train --run-name R [--resume [PATH|latest]] [--epochs N] [--max-steps N]
+[--max-hours H] [--micro-batch B] [--grad-checkpointing] [--limit-docs N] [--checkpoint-every N]
+[--log-every N] [--eval-every N] [--keep-checkpoints K] [--fresh]`. After the run:
+
+```bash
+$PY src/pretrain.py plot --run-name large-24L      # -> reports/pretrain_curves.png
+$PY src/pretrain.py export --run-name large-24L    # only if stopped early: builds final/ from the latest checkpoint
+```
+
+How it stops (every path writes a checkpoint first; checkpoints are atomic tmp-dir + rename):
+
+| Event | Behaviour | Exit code |
+|---|---|---|
+| Normal end | `final/` written | 0 |
+| Ctrl-C / SIGTERM | save at the next optimizer-step boundary | 0 |
+| `--max-hours` reached | save | 0 |
+| **VRAM guard**: this process's reserved peak > `vram.hard_cap_gb` (20 GiB) | save, stop before a real OOM | 3 |
+| **CUDA OOM** (e.g. the desktop grabs VRAM) in forward/backward, optimizer step or eval | partial step discarded, data cursor rewound, save | 3 |
+| 5 consecutive non-finite losses (each such step is skipped) | status `diverged`, save | 4 |
+| Config/plan mismatch on resume (model shape, `pretrain:` LR/betas/eps/decay/clip/masking/schedule, seed, effective batch, epochs, data) | refuses before training | 2 |
+
+Checkpoints every 500 steps and at every epoch end (keep 3, ~3.8 GB each) in
+`checkpoints/pretrain/large-24L/step-NNNNNNN/`; val MLM loss and accuracy on 64 fixed held-out windows every
+500 steps in `metrics.csv`. If val MLM loss rises, stop the run (Ctrl-C) and use `pretrain.py export`.
+
+### 5.5 Synthetic supplement (`src/make_synthetic.py`) — already built
+
+```bash
+$PY src/make_synthetic.py build [--n 2000] [--out PATH] [--seed S]   # -> data/processed/synthetic.parquet + _report.json
+$PY src/make_synthetic.py show --k 6
+```
+
+### 5.6 Fine-tuning (`src/finetune.py`) — after pretraining, GPU free
+
+Baseline (config defaults: 3 classes, cross-entropy over the partial-label NLL, 3 epochs, batch 32 = micro-batch
+16 x 2, LR 3e-5, early stopping on val F1 of the unwanted roll-up, patience 2):
+
+```bash
+$PY src/finetune.py train --run-name ft-ce --init pretrained:checkpoints/pretrain/large-24L/final
+```
+
+`ft-ce` is the cross-entropy baseline. The queue ran all six runs of section 9 on 2026-10-09 (03:36-10:02);
+**`ft-focal` won on the validation selection metric** (val F1 of the unwanted roll-up 0.9951 vs 0.9929 for
+`ft-ce`, val three-way macro-F1 0.9869 vs 0.9782) and is therefore the run evaluated at top level, exported
+and deployed. Every example in sections 5.7-5.9 uses it.
+
+**The queue (`scripts/run_experiments.sh`)** is how the experiments are meant to run: it waits for
+`checkpoints/pretrain/large-24L/final/model.safetensors`, renders `pretrain.py plot`, then trains the six runs
+of section 9 one after another with `--eval-every 1250`, plots each to `reports/finetune_curves_<run>.png`,
+evaluates each `best/` on both holdouts into `reports/eval_<run>/` (marker `reports/eval_done_<run>`), and
+finally evaluates + exports the main run (`MAIN_RUN`, default `ft-ce`) into the **top-level**
+`reports/metrics.json` / `metrics_test3.json` that section 2.1 quotes, `model_export/<MAIN_RUN>/` and `app/`.
+The top-level evaluate + export were re-run by hand for `ft-focal` after the queue finished (section 9).
+A run whose `best/` exists is skipped, so re-running the script continues after a crash.
+
+```bash
+tmux new-session -d -s phish-finetune 'scripts/run_experiments.sh 2>&1 | tee logs/experiments_$(date +%Y%m%d-%H%M%S).out'
+```
+
+Environment: `PRETRAIN_RUN` (default `large-24L`), `EVAL_EVERY` (default 1250), `MAIN_RUN` (default `ft-ce`).
+
+The instructor's dials (all flags of `train`):
+
+| Dial | Flag | Values |
+|---|---|---|
+| Loss function | `--loss` | `cross_entropy` / `weighted_cross_entropy` (inverse-frequency) / `focal` (`--focal-gamma 2.0`) |
+| Epochs | `--epochs` | e.g. 1-5; early stopping may end sooner |
+| Number of layers | `--n-layers N` | keep the first N pretrained layers (e.g. 12 of 24) |
+| Frozen layers | `--freeze-layers K` | freeze embeddings + first K layers |
+| Synthetic data | `--synthetic on\|off` | appends the 2,000-row supplement to train only |
+| Init | `--init scratch` / `--init pretrained:<dir>` | scratch = ablation showing what pretraining buys |
+| Others | `--lr`, `--batch-size`, `--micro-batch`, `--num-labels 2\|3`, `--seed`, `--eval-every`, `--log-every`, `--eval-batch-size` | |
+| Dev only | `--limit-rows N`, `--max-steps N`, `--cpu` | not evidence |
+
+Always pass `--init` explicitly: without it the script takes the newest `checkpoints/pretrain/*/final` by
+modification time, and `checkpoints/pretrain/dev/final` (a tiny self-test model) exists.
+
+```bash
+$PY src/finetune.py train --run-name ft-focal --resume     # continue <run>/last after a crash or Ctrl-C
+$PY src/finetune.py train --run-name ft-focal --overwrite  # discard an existing run dir of that name
+$PY src/finetune.py list                                   # experiments.jsonl as a markdown table
+$PY src/finetune.py plot --run-name ft-focal               # -> reports/finetune_curves.png (`--out PATH` to rename)
+```
+
+On resume, `--init/--n-layers/--freeze-layers/--batch-size/--seed/--num-labels` are refused; `--lr` is
+re-applied to the restored optimizer. Output: `checkpoints/finetune/<run>/{best,last}/`, `metrics.csv`,
+`epochs.csv`. The test files are refused by name and by `source` column.
+
+### 5.7 Evaluation (`src/evaluate.py`) — the only script that opens the holdouts
+
+```bash
+$PY src/evaluate.py run --checkpoint checkpoints/finetune/ft-focal/best --split test,test3
+$PY src/evaluate.py run --checkpoint checkpoints/finetune/ft-focal/best --split val   # model selection view, `_val` suffix
+```
+
+The queue runs this per experiment with `--run-name <run> --out-dir reports/eval_<run>`; only the main run is
+written to the top-level `reports/` (section 2.1).
+
+Writes `reports/metrics.json`, `metrics_test3.json`, `latency.json` (batch-1 `timeit.default_timer` around
+tokenize + forward + softmax, GPU bf16 and CPU fp32, next to the 74 ms benchmark), `confusion_matrix*.png`,
+`confusion_matrix3_test3.png`, `roc_curve*.png`, `pr_curve*.png`, `error_analysis*.md` (top false negatives /
+positives, phishing<->spam confusions, length-confound check, credential-lure slice), one line in
+`reports/experiments.jsonl` and one RUN_LOG.md entry. Flags: `--threshold`, `--batch-size`,
+`--latency-samples`, `--cpu-threads`, `--max-len`, `--cpu`, `--out-dir`, `--data-dir`, `--run-name`,
+`--limit-rows` (dev; no RUN_LOG entry), `--no-run-log`, `--finetune-run-dir` / `--pretrain-run-dir` (extra
+`eval_*_curves.png`). The binary metrics use P(unwanted) at 0.5; the three-way decision is roll-up first,
+then argmax(phishing, spam), so the 3x3 matrix collapses exactly onto the binary one (plain argmax-of-3 is
+reported alongside).
+
+### 5.8 Export (`src/export.py`)
+
+```bash
+$PY src/export.py run --checkpoint checkpoints/finetune/ft-focal/best
+```
+
+Writes `model_export/ft-focal/` (`model.safetensors` bf16 ~675 MB, `config.json` with label names, metrics and
+dials, `tokenizer.json`, `README.md` model card, `export_manifest.json`) and copies `model.py` (sha1-checked)
+plus the three weight files into `app/`. It refuses if `reports/metrics*.json` name a different checkpoint, so
+run 5.7 on the same `best/` first (`--allow-metrics-mismatch` overrides and records it). Other flags:
+`--out-dir`, `--app-dir`, `--dtype {bf16,fp32}`, `--metrics-json`, `--metrics3-json`, `--pretrain-config`.
+
+Local app check before uploading: `CUDA_VISIBLE_DEVICES="" $PY app/app.py` (CPU; drop the prefix once the GPU
+is free: `app.py` moves the model to CUDA at import time whenever a GPU is visible).
+
+### 5.9 Upload (`scripts/upload_to_hf.py`) — Nathan only
+
+```bash
+$PY scripts/upload_to_hf.py --model-dir model_export/ft-focal --bundle-weights --check-auth   # dry run (default): prints the plan, no writes
+$PY scripts/upload_to_hf.py --model-dir model_export/ft-focal --bundle-weights --yes          # live (what was run on 2026-10-12)
+```
+
+The dry run lists every file, size, repo, visibility, the hardware flavour and any blocking problem (a missing
+export, a missing `app/model.py`, or `HF_TOKEN_FOR_SPACE` unset when the weights are not bundled). Other flags:
+`--bundle-weights` (weights inside the Space, no secret needed; the mode used), `--skip-secret`, `--hardware`,
+`--model-repo`, `--space-repo`, `--private/--no-private`, `--commit-message`. `--yes` is only ever run with
+Nathan's explicit approval; the one live run happened on 2026-10-12 11:35 (RUN_LOG 24) after his approval of
+2026-10-08. Lesson from that run: the Hub validates the Space card's YAML front matter at upload time and the
+dry run does not, so a colon inside an unquoted `short_description` failed the first attempt; the value is now
+quoted and the model-repo half of the first attempt was simply reused (LFS blobs dedupe by content hash).
+
+### 5.10 The notebook
+
+`FinalProject_PhishingDetector.ipynb` (project root; generated by `scripts/build_notebook.py`, last executed top
+to bottom on the CPU on 2026-10-12: 36 code cells, no errors, every artifact present) is the presentation
+walk-through. It mirrors the instructor's
+`reference/FashionMNIST_Gradio_Full.ipynb` step for step, with the nine headers quoted verbatim: "Step 1: Install
+Dependencies & Setup Device", "Step 2: Download Data & Create DataLoaders", "Step 3: Define the Neural Network",
+"Step 4: Train the Model", "Step 5: Test Inference Speed and Save the Model", "Step 6: Create the App Files
+(Optional)", "Step 7: Launch the Gradio Interface", "Step 8: Authenticate with Hugging Face", "Step 9: Upload
+the App to Hugging Face Spaces". It **loads the artifacts above instead of training**: the contract is that it
+runs top to bottom on CPU and prints "not available yet — produced by <command>" for anything missing; with the
+2026-10-09 artifacts in place it shows the real numbers, curves, confusion matrices and the `ft-focal` export.
+
+```bash
+CUDA_VISIBLE_DEVICES="" .venv/bin/jupyter lab
+CUDA_VISIBLE_DEVICES="" .venv/bin/jupyter nbconvert --to notebook --execute --inplace FinalProject_PhishingDetector.ipynb
+```
+
+---
+
+## 6. Config dials (`config.yaml`)
+
+| Section | What it controls |
+|---|---|
+| `seed`, `paths` | global seed 42; every relative path is resolved against the project root |
+| `model` | encoder shape (24 / 1024 / 16 / 4096 / 32768 / 512), dropout, pre-LN, tied MLM head, gradient checkpointing |
+| `tokenizer` | byte-level BPE, vocab 32,768, min frequency 2, specials `[PAD] [UNK] [CLS] [SEP] [MASK]`, training corpora |
+| `data` | sources, exclusions, holdout source (TREC-06), alnum minimums, dedup keys and shingle parameters, val fraction, `synthetic`, `threeway` (label3 by provenance, Champa mirror, the rejected `bigshp` set with its verdict, holdout3 carving) |
+| `vram` | 14-18 GiB target band, 20 GiB hard cap (GiB, compared with `torch.cuda.max_memory_reserved()`), allocator config |
+| `pretrain` | bf16, packing, effective batch 256, micro-batch start 24 + candidates, 12 epochs, AdamW LR 1.5e-4 / betas (0.9, 0.98) / eps 1e-6 / decay 0.01 / clip 1.0, warmup, cosine to 10%, 15% masking 80/10/10, checkpoint/log/eval cadence |
+| `finetune` | `num_labels: 3`, epochs, batch / micro-batch, LR 3e-5, warmup + linear decay, loss and class weights, focal gamma, early stopping metric + patience, frozen layers, padding multiple |
+| `eval` | metric list, credential-lure slice, latency samples, threshold 0.5, test3 file, label contract (asserted against the code), top-k confusions, model-card metrics source |
+| `hf` | owner, model and Space repo ids, private flag, Gradio 6.29.1, Python 3.12.12, `zero-a10g`, Space torch 2.13.0, safetensors bf16 |
+
+**Rule during a live pretraining run:** do not edit `model:` or the `pretrain:` keys that shape the plan
+(`lr`, `betas`, `eps`, `weight_decay`, `grad_clip`, `mlm_probability`, `mask_replace_probs`, `schedule`,
+`effective_batch_sequences`, `warmup_steps`, `warmup_fraction`), `seed`, or anything that changes
+`pretrain.parquet` / `tokenizer.json`. Each is stored in the checkpoint and a resume refuses a mismatch. This is
+why `num_labels` lives under `finetune:`. `finetune:`, `eval:` and `hf:` can be edited at any time.
+
+---
+
+## 7. Data provenance and licences
+
+| Source | Used for | Licence / terms | Notes |
+|---|---|---|---|
+| `puyang2025/seven-phishing-email-datasets` @ b7f3f8d9 (203,017 rows) | labelled legit + unwanted: TREC-05/06/07, CEAS-08, Enron-Spam, SpamAssassin | Hub card "other"; upstream Champa, Rabbi & Zibran, "Seven Phishing Email Datasets", figshare 25432108, CC BY 4.0 | Its own train/test split leaks (5.1% exact duplicates) and is ignored. Ling (2,859 academic list rows) excluded. Label 1 = "unwanted": these are spam-track corpora with no phishing/spam distinction. |
+| `corbt/enron-emails` @ cfc06c75 (517,401 rows) | MLM pretraining only | CMU Enron corpus (FERC public release); no formal licence; CMU asks users to respect privacy | 52.7% exact duplicates removed, quoted/forwarded history stripped -> 226,273 docs. |
+| `kudzaiprichard/aura-phishing-email-corpus` @ 539567ce: `Nazario.csv`, `Nigerian_Fraud.csv` | phishing class (1,507 + 2,504 rows kept) | CC BY 4.0; byte-identical mirror of Champa, Rabbi & Zibran, "Phishing Email Curated Datasets", Zenodo, doi:10.5281/zenodo.8339691 | Nazario phishing archive and advance-fee (419) fraud, 1995-2022. 235 Nigerian rows near-duplicating TREC-06 spam and 13 exact matches removed to keep the test set clean. |
+| `data/processed/synthetic.parquet` (2,000 rows) | optional training supplement | generated by `make_synthetic.py`; fictional names, `.example`/`.invalid` domains, RFC 5737 IPs, 555-01xx phones | 8 BEC / Microsoft 365 / invoice lure categories vs 8 ordinary business categories; shared filler pools and a token-purity audit against template shortcuts. Never in val or test, never presented as real mail. |
+| Rejected: `locuoco/the-biggest-spam-ham-phish-email-dataset-300000` | nothing | MIT (Kaggle re-upload) | 365,448 rows, 51.5% internal duplicates, a union of corpora we already hold; its "phish" label agrees with provenance only 60.5%; **36,020 rows are copies of our TREC-06 test set**; phish rows went through a stopword-stripping pipeline the others did not (a label-proxy artifact). Not one row ingested. `docs/research/bigshp_research.md`. |
+| Rejected: `it4lia/PhishingEmailCuratedDatasets_Cleaned` | nothing | CC BY 4.0 | no body text (subjects/headers + 53 numeric features). |
+| Rejected: `zefang-liu/phishing-email-dataset` | nothing | LGPL-3.0 | lowercased re-packaging, 91.9% contained in the primary set, "enron" token shortcut. |
+
+Details and evidence: `docs/research/research_dataset-schemas.md`, `docs/research/verify_data.md`,
+`data/processed/data_report.md`. The exported model card states that phishing/spam labels follow corpus
+provenance.
+
+---
+
+## 8. Leakage policy
+
+- **TREC-06 is the binary test set and nothing upstream has seen it.** It is excluded from tokenizer training,
+  the MLM corpus and fine-tuning. It is Enron-clean (5 rows mention "enron", 0 enron.com addresses); TREC-05 ham
+  and Enron-Spam ham are 86-89% inside the pretraining corpus, so neither could be the test set. Model input is
+  subject + body only because TREC rewrote the spam headers.
+- **Decontamination.** Exact keys on normalised text (full, 300-char prefix, subject+body prefix, subject+body
+  full) plus near-duplicates (>= 3 shared 50-char shingles over the first 1,500 chars). 1,652 training rows that
+  near-duplicate TREC-06 templates were removed; 85 pretraining docs near-duplicating holdout emails and 4,004
+  matching val keys were removed from the MLM corpus.
+- **Train/val split** is grouped by near-duplicate cluster and stratified by source x label, so no template
+  straddles train and val.
+- **test_holdout3** rules: legitimate rows only from CEAS-08, TREC-07, SpamAssassin (never Enron, which overlaps
+  pretraining); spam from SpamAssassin and Enron-Spam spam folders; phishing from Nazario + Nigerian_Fraud.
+  Rows come from finetune_val (never seen by the tokenizer) or the new sources (never seen by anything); whole
+  clusters move or none; any cluster with a key or shingle link to the pretraining corpus or train is excluded.
+- **Asserted zeros** (recomputed independently from the written files by `check` / `check3`; a non-zero count
+  fails the build): holdout vs train / val / pretrain keys and shingle links; train vs val full-key rows; val vs
+  pretrain keys; holdout source rows in train/val; holdout3 vs train / val / pretrain keys and shingle links;
+  holdout3 vs TREC-06 keys; doc_id shared between splits; label3 contract violations. All are 0
+  (`data/processed/data_report.md`). Counted but allowed: 5,336 val-to-pretrain near-dup links (val is only
+  used for model selection), 102,100 train-to-pretrain key overlaps (Enron ham).
+- **Code guards.** The tokenizer refuses val/test files by name with an exact-hash tripwire (0 collisions);
+  `finetune.py` refuses the holdout by file name and `source` column; `pretrain.py` never opens it;
+  `evaluate.py` is the only reader.
+
+---
+
+## 9. Experimentation record
+
+- **`reports/experiments.jsonl`**: `finetune.py train` appends one JSON line per run (dials: init, layers,
+  frozen layers, epochs planned/run, loss, focal gamma, LR, batch, class weights, synthetic rows, seed,
+  selection metric; parameter counts; steps, seconds, peak VRAM; best step; val metrics) and `evaluate.py`
+  appends one line per invocation with the test metrics. `finetune.py list` renders it as the markdown table for
+  the presentation. Generated from evidence, never typed up afterwards.
+- **Contents**: the six real runs below (2026-10-09) plus one evaluate line per holdout evaluation. The dev
+  self-test rows from 2026-10-08 were deleted before the queue ran (RUN_LOG 9). `reports/finetune_curves.png` is
+  the `ft-focal` curve, `reports/finetune_curves_<run>.png` the others, `reports/pretrain_curves.png` the
+  `large-24L` curve.
+- **Pretraining** is one run (`large-24L`); its curve is `metrics.csv` -> `pretrain.py plot`.
+- **Fine-tune queue** (`scripts/run_experiments.sh`, section 5.6; one architecture, one dial changed per run
+  against the baseline). It ran unattended on 2026-10-09 from 03:36 to 10:02 (six runs, each evaluated on both
+  holdouts). The baseline `ft-ce` was allowed 3 epochs; the five ablations 2 epochs each for run-time reasons;
+  early stopping (patience 2 on the validation unwanted-F1, evaluated every 1,250 steps) ended five of the six
+  before their epoch budget. All start from `checkpoints/pretrain/large-24L/final` except `ft-scratch`.
+
+| Run | Flags | Epochs | Question it answers |
+|---|---|---|---|
+| ft-ce | `--loss cross_entropy` | 3 | reference point: pretrained 24L, cross-entropy, synthetic off |
+| ft-weighted | `--loss weighted_cross_entropy` | 2 | class imbalance (3,150 phishing vs 81,438 legit fully-labelled rows) |
+| ft-focal | `--loss focal` (gamma 2.0 from config) | 2 | hard-example weighting |
+| ft-synthetic | `--loss cross_entropy --synthetic on` | 2 | does the BEC/M365 supplement move anything (reported even if it does not) |
+| ft-12L | `--loss cross_entropy --n-layers 12` | 2 | number of layers (half the stack, ~2x faster inference) |
+| ft-scratch | `--loss cross_entropy --init scratch` | 2 | what MLM pretraining buys |
+
+### 9.1 Results (validation = selection; holdouts = reporting)
+
+Selection view (validation split, 15,284 rows; the queue's early-stopping metric is the unwanted-F1 column):
+
+| Run | Loss / dial | Epochs run / planned | Stopped by | Best step | Train min | Peak GiB | Val F1 unwanted (selection) | Val macro-F1 3-way | Val P / R phishing |
+|---|---|---|---|---|---|---|---|---|---|
+| ft-ce | cross_entropy | 1.25 / 3 | early_stopping | 5000 | 51 | 13.13 | 0.9929 | 0.9782 | 0.939 / 0.997 |
+| ft-weighted | weighted_cross_entropy | 2.00 / 2 | epochs_done | 10000 | 84 | 13.13 | 0.9937 | 0.9775 | 0.963 / 0.997 |
+| **ft-focal** | focal (gamma 2.0) | 2.00 / 2 | early_stopping | 8750 | 84 | 13.13 | **0.9951** | 0.9869 | 0.985 / 0.988 |
+| ft-synthetic | cross_entropy, synthetic on | 1.23 / 2 | early_stopping | 5000 | 52 | 13.13 | 0.9927 | 0.9665 | 0.881 / 0.997 |
+| ft-12L | cross_entropy, 12 layers | 1.75 / 2 | early_stopping | 6250 | 35 | 6.97 | 0.9925 | 0.9774 | 0.957 / 0.982 |
+| ft-scratch | cross_entropy, scratch init | 1.50 / 2 | early_stopping | 5009 | 61 | 13.13 | 0.9875 | 0.9538 | 0.938 / 0.941 |
+
+Held-out view (TREC-06: 16,302 rows, whole corpus unseen; test_holdout3: 3,220 rows, 1,999 / 521 / 700). Evaluated once per run after training; not used for selection:
+
+| Run | TREC-06 recall | precision | F1 | FPR | ROC-AUC | PR-AUC | recall @ 1% FPR | test3 F1 unwanted | test3 macro-F1 | F1 phishing | F1 spam |
+|---|---|---|---|---|---|---|---|---|---|---|---|
+| ft-ce | 0.9404 | 0.8610 | 0.8990 | 4.8% | 0.9875 | 0.9626 | 0.7745 | 0.9865 | 0.9822 | 0.9769 | 0.9779 |
+| ft-weighted | 0.9818 | 0.8307 | 0.9000 | 6.3% | 0.9943 | 0.9833 | 0.8767 | 0.9862 | 0.9852 | 0.9875 | 0.9767 |
+| **ft-focal** | 0.9741 | 0.8794 | **0.9243** | 4.2% | 0.9946 | 0.9848 | 0.8955 | 0.9864 | **0.9867** | 0.9913 | 0.9771 |
+| ft-synthetic | 0.9679 | 0.8402 | 0.8995 | 5.8% | 0.9908 | 0.9716 | 0.7871 | 0.9873 | 0.9814 | 0.9772 | 0.9749 |
+| ft-12L | 0.9461 | 0.8120 | 0.8739 | 6.9% | 0.9836 | 0.9513 | 0.6608 | 0.9823 | 0.9807 | 0.9807 | 0.9721 |
+| ft-scratch | 0.9165 | 0.7752 | 0.8400 | 8.3% | 0.9755 | 0.9456 | 0.7773 | 0.9767 | 0.9634 | 0.9574 | 0.9470 |
+
+What the dials say: focal loss is the clear winner (fewer misses on TREC-06 at a lower false-alarm rate, and
+the best phishing-vs-spam separation); inverse-frequency class weights buy recall at the cost of precision;
+the 2,000-row synthetic supplement does not help (reported, as promised, even though it does not move anything);
+halving the stack to 12 layers costs ~5 F1 points on TREC-06 for ~2x faster inference; and scratch
+initialisation is 8 F1 points behind the same model with MLM pretraining, which is what pretraining buys.
+`ft-focal` was selected on validation alone, before the holdouts were read, and is the run evaluated at top
+level (RUN_LOG 22), exported to `model_export/ft-focal/` + `app/` and uploaded (section 10).
+
+Not run, optional if time allows: `--freeze-layers 12` (forgetting vs cost), an epochs sweep (`--epochs 1` /
+`5`), `--num-labels 2` (binary baseline comparable with the binary-era plan).
+
+---
+
+## 10. Deployment
+
+| Item | Value |
+|---|---|
+| Model repo | `BuildITSystems/phishing-detector-encoder` (private): `model.safetensors` (bf16), `config.json`, `tokenizer.json`, `README.md` model card |
+| Space | `BuildITSystems/phishing-detector-demo` (private), Gradio SDK 6.29.1, hardware `zero-a10g` (ZeroGPU) |
+| Space stack | **torch 2.13.0 on Python 3.12.12**: ZeroGPU supports torch 2.8.0-2.13.0 and Python 3.12.12 / 3.10.13 only, and no torch.compile. Training uses 2.14.1 on 3.14; safetensors makes the weights portable. Gradio is pinned by `sdk_version` in `app/README.md`, not in `requirements.txt`. |
+| Weights at startup | The three weight files are **bundled into the Space** (`upload_to_hf.py --bundle-weights`, the mode used), so no secret is needed and the Space works even while the model repo is private. `app.py` can alternatively download them from the repo named by the `PHISH_MODEL_REPO` variable with an `HF_TOKEN` read-token secret. |
+| UI | Subject + body in; 3-class `gr.Label`, "Prediction Time (s)" (`timeit.default_timer`), gradient x input token highlights, rule-based cues shown for context only (they never feed the probability). |
+
+**Before class: make both repos public** (the instructor needs the model link and a Space that opens without
+an invite), then open the Space once so ZeroGPU is warm. `create_repo(exist_ok=True)` does not change the
+visibility of an existing repo, so do it in each repo's Settings page on huggingface.co, or:
+
+```bash
+.venv/bin/python -c "from huggingface_hub import HfApi; a=HfApi(); a.update_repo_settings('BuildITSystems/phishing-detector-encoder', private=False); a.update_repo_settings('BuildITSystems/phishing-detector-demo', repo_type='space', private=False)"
+```
+
+`app/README.md` (the Space card) already links the model repo.
+
+---
+
+## 11. Instructor requirement mapping
+
+| Requirement (`reference/INSTRUCTOR_REQUIREMENTS.md`) | Where it is satisfied |
+|---|---|
+| Classifier, not regressor | 3-class softmax classifier + binary roll-up (section 1) |
+| Dataset from `torchvision.datasets` | **Deviation, agreed text variant**: there is no torchvision email dataset. Public Hugging Face datasets loaded with `datasets` (section 7); the PyTorch data pipeline is kept (DataLoader below). |
+| Problem statement: input, classes, challenge | Section 1 |
+| Algorithm and architecture, >= 2 layers | 24-layer transformer encoder, `src/model.py`; justification in `docs/research/research_model-math.md` and RUN_LOG 0 |
+| Loss function | Partial-label NLL (= cross-entropy for fully-labelled rows) with `--loss cross_entropy / weighted_cross_entropy / focal`; MLM cross-entropy on masked tokens for pretraining |
+| Optimizer | AdamW (fused, no decay on biases/LayerNorm), grad clip 1.0, bf16 autocast |
+| Epochs, early stopping / LR scheduling | Fine-tune: early stopping on val F1 (patience 2), warmup + linear decay. Pretrain: warmup + cosine to 10%, val MLM loss every 500 steps |
+| Justify choices | This README, RUN_LOG.md, `docs/research/` |
+| `DataLoader` with batching, shuffling, preprocessing | `finetune.make_loader`: `torch.utils.data.DataLoader` with a seeded length-bucketed shuffled `batch_sampler` and a padding collate; tokenisation as the preprocessing transform (RUN_LOG 8: moved from a numpy sampler on 2026-10-08 13:55, losses verified identical, explicit `generator` to keep the dropout stream unchanged) |
+| Evaluate on an unseen test set with standard metrics | `evaluate.py` on TREC-06 and test_holdout3: accuracy, precision, recall, F1, ROC-AUC, PR-AUC, confusion matrices (section 2.1) |
+| Inference timer, seconds per single prediction | `evaluate.py` -> `reports/latency.json` (GPU and CPU, batch 1); `smoke_test.py`; the app's "Prediction Time (s)" |
+| Gradio UI: input -> label + inference time | `app/app.py` (text input instead of an image upload) |
+| HF Spaces with `app.py`, weights, `requirements.txt` | Section 10; `upload_to_hf.py` |
+| Class: dataset, trained model, performance results | Sections 2, 7; `reports/` |
+| Class: model on Hugging Face, link to instructor | `https://huggingface.co/BuildITSystems/phishing-detector-encoder` (make public first) |
+| Class: live Gradio app on a Space, not gradio.live | `https://huggingface.co/spaces/BuildITSystems/phishing-detector-demo` |
+| Class: Space has app.py, model.py, requirements.txt (pinned torch/gradio) + weights | `app/`; torch pinned in requirements.txt, gradio pinned by `sdk_version` |
+| Class: show experimentation (layers, epochs, loss), record each run | Section 9, `reports/experiments.jsonl`, `finetune.py list`, RUN_LOG.md |
+| Class: prediction time per sample vs 0.074 s, GPU and CPU | Section 2.1: 10.1 ms GPU bf16, 459 ms CPU fp32 per email (`reports/latency.json`) |
+| Class: presentation covers data, algorithm, performance, challenges | RUN_LOG.md (OOM tuning, leakage, rejected datasets, reviewer-found bugs), sections 2, 7, 8, 12 |
+| Class: synthetic data allowed | `make_synthetic.py`, `--synthetic on`, reported with and without |
+| Class: one strong model, not an architecture comparison | One architecture (24L encoder); the six runs of section 9 vary training dials only, and `ft-focal` is the one deployed |
+| Notebook in the instructor's 9-step pattern | Section 5.10 |
+
+---
+
+## 12. Known limitations and ethics
+
+- **Defensive use only.** It flags mail for a human; it is not a verdict, not a replacement for SPF/DKIM/DMARC,
+  sender reputation, link or attachment scanning, and must not be used to tune lures against.
+- **Old corpora.** Training mail is English, 1995-2022 with the bulk 1998-2008 (Enron, TREC, CEAS,
+  SpamAssassin). Modern lures (MFA fatigue, QR codes, chat-app phishing) and non-English mail are out of
+  distribution; the synthetic supplement only nudges this.
+- **Spam vs phishing follows provenance**, not per-message review: phishing = Nazario + Nigerian fraud +
+  synthetic lures; spam = SpamAssassin and Enron-Spam spam folders. Only 3,150 real phishing rows carry the
+  distinction; 62,805 TREC/CEAS positives train as "unwanted" only. Trust the roll-up first.
+- **Length confound.** Legitimate emails average about twice the tokens of unwanted ones in every split
+  (TREC-06: 842 vs 283). Truncation at 512 tokens blunts it; `evaluate.py` checks it in the error analysis.
+- **Truncation.** About 22% of emails exceed 512 tokens (~1,800 characters) and are judged on their opening.
+- **Test-set caveat.** About 47% of TREC-06 ham is academic mailing-list traffic, which is not typical business mail.
+- **CPU latency.** 699 ms per email on CPU vs the 74 ms benchmark; the benchmark is met on GPU only.
+- **Privacy.** Enron mail is real correspondence released by FERC; nothing pasted into the Space is stored,
+  but do not paste secrets or client content.
+
+---
+
+## 13. Troubleshooting
+
+| Symptom | Cause and fix |
+|---|---|
+| Pretraining exits 3 ("CUDA OOM" or "VRAM guard") | Another process took VRAM (browser, game, a second job) or the 20 GiB guard tripped. A checkpoint was saved. Check `nvidia-smi`, close the culprit, then `RESUME=1 MICRO_BATCH=24 scripts/launch_pretrain.sh`. The run's own peak is ~17.9 GiB reserved and the desktop adds ~2 GiB, so total device use sits near 20 GB. If it recurs, resume directly with `$PY src/pretrain.py train --run-name large-24L --resume --micro-batch 24 --grad-checkpointing` (8.06 GiB measured, ~25% slower). Accumulation = round(256 / micro-batch), so of the micro-batches that fit in 24 GB only 24 (x11) and 22 (x12) keep the effective batch at 264; 16 gives 256 and is refused on resume. |
+| Resume refused, exit 2, "'effective_batch' differs" | `launch_pretrain.sh` defaults to `MICRO_BATCH=16`; pass `MICRO_BATCH=24`. |
+| Resume refused, "'pretrain'/'schedule'/'data' differs" | `config.yaml` `pretrain:`/`model:`/`seed` was edited, or `pretrain.parquet` / `tokenizer.json` changed. Revert the edit; see section 6. |
+| `launch_pretrain.sh`: "tmux session already exists" | The finished pane waits for Enter: `tmux attach -t phish-pretrain` and press Enter, or `tmux kill-session -t phish-pretrain` once the trainer has exited. |
+| Fine-tune crashed or was stopped | `$PY src/finetune.py train --run-name <run> --resume` (restores optimizer, RNG and batch position). |
+| Fine-tune picked the wrong pretrained model | Pass `--init pretrained:checkpoints/pretrain/large-24L/final`; the default is the newest `*/final` by mtime and `dev/final` exists. |
+| Install resolves a Python 3.15 interpreter | torch.compile, triton 3.8.0 and `spaces` all exclude 3.15. Recreate with `uv venv --python 3.14.7 .venv`. |
+| Install fails on fsspec | datasets 5.1.0 caps `fsspec<=2026.7.0`; keep `fsspec==2026.7.0` from requirements-train.txt and never upgrade it alone. |
+| Install pulls cu132 wheels | `--torch-backend=auto` was used; reinstall with `--torch-backend=cu130`. |
+| huggingface_hub 2.x conflict | tokenizers 0.23.2 needs `<2.0`; keep 1.33.0. |
+| `export.py` refuses: metrics name another checkpoint | Run `evaluate.py run --split test,test3` on the same `best/` first. |
+| Space build fails on torch | ZeroGPU allows torch <= 2.13.0 and Python 3.12.12; keep `app/requirements.txt` and the `python_version` front matter as they are. |
+| Space starts but cannot load weights | `HF_TOKEN` secret missing or not a read token for the model repo, or `PHISH_MODEL_REPO` variable wrong; re-run the upload with `HF_TOKEN_FOR_SPACE` set, or use `--bundle-weights`. |
+| Any Python command while pretraining runs | Prefix with `CUDA_VISIBLE_DEVICES=""`; never run `smoke_test.py`, a GPU fine-tune or GPU evaluation alongside it. |
diff --git a/presentation/RUN_LOG.md b/presentation/RUN_LOG.md
new file mode 100644
index 0000000000000000000000000000000000000000..a10b7f16e23a4e7a512f661e7d74889dfb7df2f6
--- /dev/null
+++ b/presentation/RUN_LOG.md
@@ -0,0 +1,931 @@
+# RUN_LOG — BITS Phishing Detector
+
+Running record of what was run, with what settings, what happened, and what was fixed.
+Newest entries at the bottom. This becomes the "challenges" section of the presentation.
+
+---
+
+## 0. 2026-10-07 — Plan confirmation, environment, pins
+
+**Machine.** Fedora 44 KDE, RTX 3090 Ti 24 GB (sm_86), driver 615.71.09 (CUDA 13.4), Ryzen 9 5950X,
+40 GB RAM, 741 GB free. System Python 3.14.7, uv 0.12.5. ~1.9 GB VRAM already used by the desktop.
+
+**Decision: Python 3.14 stays.** Checked live: torch 2.14.1 and every other pinned package ship
+cp314 (or abi3 / pure) Linux wheels; torch.compile is allowed on 3.14 (only 3.15 raises); triton 3.8.0
+requires <3.15. No Python update needed. (uv also downloaded a managed CPython 3.12.14 as a side
+effect of dry-resolving the Space requirements; harmless, kept for testing the Space stack locally.)
+
+**Decision: torch 2.14.1, CUDA 13.0 build.** The plain PyPI Linux wheel *is* the cu130 build
+(cuda-toolkit 13.0.3, cuDNN 9.24, NCCL 2.30.7). Driver 615 (CUDA 13.4) runs it by backward compatibility.
+Install with `uv pip install --torch-backend=cu130`; never `--torch-backend=auto` (uv 0.12.5 would pick
+cu132 on this driver).
+
+**Constraint found: ZeroGPU pins the Space stack.** ZeroGPU supports torch 2.8.0 – 2.13.0 and Python
+3.12.12 / 3.10.13 only, and does not support torch.compile. So the Space uses torch 2.13.0 on Python
+3.12.12 while training uses 2.14.1 on 3.14. Weights are safetensors, so this is fine. Also: a private
+model repo cannot be preloaded by the Space; weights download at startup with an HF_TOKEN secret.
+
+**Constraint found: huggingface_hub must stay on 1.x.** tokenizers 0.23.2 requires huggingface-hub<2.0,
+so huggingface_hub 1.33.0 (ships the `hf` CLI) everywhere.
+
+**Data problems found (before downloading anything):**
+- `it4lia/PhishingEmailCuratedDatasets_Cleaned` has **no body text** (subjects/headers + 53 numeric
+  features). Unusable for a text model. Excluded.
+- `zefang-liu/phishing-email-dataset` is a lowercased re-packaging already 91.9% contained in
+  `puyang2025/seven-phishing-email-datasets` (one 17 MB row, 533 literal "empty" rows, and 20% of its
+  "Safe" rows contain the token "enron" vs 0% of phishing: a shortcut). Excluded.
+- The primary set's own train/test split leaks: 5.1% of test rows are exact duplicates of train rows,
+  ~40% have a near-duplicate. We build our own split.
+- Two sub-sources are Enron-derived: 89% of TREC-05 ham and 86% of Enron-Spam ham appear in the
+  `corbt/enron-emails` pretraining corpus. Neither can be the test set.
+- `corbt/enron-emails` is 52.7% exact duplicates (517,401 -> 244,496 distinct); ~117M tokens remain,
+  about 230k packed 512-token sequences per epoch, so one epoch is ~1,144 optimizer steps at batch 256.
+- No dataset separates "spam" from "phishing". Every label-1 row comes from a spam-track corpus.
+  Mapping spam->0 would leave ~0 positives. **Proposed:** positive class = "phishing/spam (unwanted)",
+  plus a credential-lure slice of the test set reported separately. PENDING Nathan's confirmation.
+
+**Decision: hold out TREC-06 entirely as the test set** (16,259 rows after dropping 141 near-empty
+rows; 12,394 legitimate / 3,865 positive = 23.8%). It is Enron-clean (5 rows mention "enron", 0
+enron.com addresses). Caveats: ~47% of its ham is academic mailing-list traffic; its spam headers were
+rewritten by TREC, so the model input is subject + body only. 1,766 training rows that near-duplicate
+TREC-06 templates get removed from the training pool.
+
+**Model math.** 24L / d1024 / 16 heads / FFN 4096 / vocab 32768 / seq 512 = 337,476,608 params
+(pretraining) / 337,443,842 (classifier). AdamW state 5.0 GiB. Micro-batch 16 without checkpointing
+is predicted at ~16.6 GiB total (7.4 GiB headroom) -> inside the 14–18 GB budget. Predicted
+throughput 12k–18k tokens/s (30–45% MFU of 80 TFLOPS bf16) -> 2.3–3.5 h per 150M-token epoch.
+Inference: ~14 ms per 512-token email on the 3090 Ti; ~0.3 s on the CPU in fp32 (the instructor's
+0.074 s/sample benchmark is met on GPU, not on CPU at full length).
+
+**Status.** Project scaffolded, venv created on 3.14.7, pinned install started (logs/install.log).
+Next: verify GPU from torch, then data_prep -> tokenizer -> model -> smoke test -> pause for numbers.
+
+**Step 2 result (22:45).** Pinned install succeeded (180 packages, all binary wheels; venv 6.1 GB).
+From inside the venv: torch 2.14.1+cu130, CUDA 13.0 build, cuDNN 9.24; device NVIDIA GeForce RTX
+3090 Ti, sm_86, 23.60 GiB, bf16 supported; SDPA Flash / memory-efficient / math backends all run
+(Flash with no mask, memory-efficient with a bool padding mask); bf16 4096^3 matmul measured
+63.9 TFLOPS (spec ~80); torch.compile works on Python 3.14 (triton built its driver stub with a
+harmless _POSIX_C_SOURCE warning); free VRAM at idle with the desktop up: 21.5 GiB. All pinned
+libraries import; config.yaml parses. `hf` CLI present, not logged in yet (Nathan to run `hf auth login`).
+
+**Agent policy (per Nathan, 21:14).** Model and effort are set per agent from here on: mechanical
+lookups -> Sonnet/low; judgment-heavy research -> Opus or Fable/medium-high; math, core code and
+adversarial verification -> Fable/high. The step-1 research run inherited Fable for all 7 agents.
+
+## 1. 2026-10-07 23:41 — Data build (data_prep.py build + check)
+
+**Run.** `.venv/bin/python src/data_prep.py build` then `check`. 2 min 20 s wall, peak RSS 6.9 GB,
+HF cache 1.5 GB (puyang2025 @ b7f3f8d9, corbt @ cfc06c75). Every asserted leakage check is 0 in
+both the build-time recount and the independent `check` over the written files.
+
+| split | rows | positives | base rate | sources |
+|---|---|---|---|---|
+| finetune_train | 157,113 | 75,675 | 48.2% | TREC-05 46,386 / TREC-07 47,233 / CEAS-08 34,178 / Enron 24,223 / Assassin 5,093 |
+| finetune_val | 17,643 | 8,642 | 49.0% | same five, grouped by near-dup cluster so no template straddles train/val |
+| test_holdout (TREC-06) | 16,302 | 3,894 | 23.9% | never touched by tokenizer, pretraining, or fine-tuning |
+| pretrain (MLM) | 226,273 docs | - | - | corbt/enron-emails after dedup + quote stripping; 408M chars |
+
+**What was removed and why.** 715 labelled rows with <20 alphanumerics (91% of them TREC-05/TREC-06
+garbage spam rows). Ling excluded (2,859 academic rows). 18 rows in 8 groups where the identical body
+carries both labels (Enron ham vs TREC-05 spam newsletters) — dropped as label noise. 6,715 exact
+duplicates in the pool. **1,652 training rows removed as near-duplicates of TREC-06 templates**
+(TREC-05 spam 863, Enron spam 456, TREC-07 spam 292): these are spam templates shared across the
+TREC years; keeping them would let the model memorise test-set templates. corbt: 517,401 rows ->
+272,905 exact duplicates removed (52.7%), 14,027 too short after quote stripping, 4,004 matching a
+val email removed, 85 near-duplicates of holdout emails removed, 35,172 overlapping fine-tune TRAIN
+emails kept (allowed: those are Enron ham). Quote stripping cut 21,832 bodies (kept 89% of all
+characters; the >=200-alnum guard left 70,907 marked bodies intact).
+
+**Deviation from the research estimate.** Holdout is 16,302 rows (research: 16,259) because the
+<20-alnum filter is applied to subject+body rather than body alone: 43 TREC-06 rows have a real
+subject and a near-empty body, which is a legitimate model input. Pretraining corpus is 408M chars
+(research: ~466M) because quote stripping and the val screen remove more than the estimate assumed.
+
+**Problem noted.** The val split has 5,336 near-duplicate links into the pretraining corpus
+(883 val emails, Enron boilerplate). Counted, not removed: val is only used for model selection
+and the holdout is the real gate. Exact val matches (4,004) were removed from pretraining.
+
+## 2. 2026-10-07 23:43 — Tokenizer (tokenizer.py train + stats)
+
+**Run.** Byte-level BPE, NFKC only (case preserved on purpose: ALL-CAPS is a spam signal), vocab 32,768,
+min_frequency 2, specials [PAD]=0 [UNK]=1 [CLS]=2 [SEP]=3 [MASK]=4. Trained on pretrain.parquet
+(226,273 docs, 408M chars) + finetune_train.parquet (157,113 docs, 278M chars) only; val and test are
+refused by name and an exact-hash tripwire against the holdout found 0 collisions. Training took 28 s
+(49 s wall incl. the tripwire), 3.6 GB RSS. Saved data/processed/tokenizer.json.
+
+| split | rows | mean tok | median | p95 | chars/tok | % > 512 | mean tok legit / positive |
+|---|---:|---:|---:|---:|---:|---:|---|
+| pretrain (20k sample) | 20,000 | 476 | 198 | 1,560 | 3.76 | 20.5% | - |
+| finetune_train | 157,113 | 487 | 227 | 1,540 | 3.63 | 24.1% | 621 / 343 |
+| finetune_val | 17,643 | 454 | 200 | 1,421 | 3.58 | 22.3% | 594 / 307 |
+| test_holdout | 16,302 | 709 | 244 | 1,254 | 3.43 | 21.9% | 842 / 283 |
+
+**Observations.** [UNK] rate is 0 by construction (full 256-byte alphabet). Round trips are exact
+except for NFKC folding (full-width "Ｍｉｃｒｏｓｏｆｔ" -> "Microsoft", a deliberate anti-evasion
+property). About 22% of emails are longer than 512 tokens and get truncated; the pretraining corpus
+is ~108M tokens at 3.76 chars/token. **Length is a confound:** legitimate emails average ~2x the
+tokens of positives in every split (holdout 842 vs 283). Truncation at 512 blunts this, and the
+error analysis must check whether the classifier leans on length. Outliers exist (one 886,635-token
+email in train) and are harmless under truncation.
+
+## 3. 2026-10-07 23:46 — Smoke test (smoke_test.py, real packed text, reports/smoke_test.json)
+
+**Model.** 337,476,608 params (MLM, tied head) / 337,443,842 (classifier) — exactly the research
+figure. 3,000 real pretraining docs -> 1,592,926 tokens -> 3,111 packed 512-token sequences; in-code
+no-leak check: 0 of 3,000 docs match a holdout email.
+
+**VRAM (bf16 autocast, fused AdamW, real gradient-accumulation pattern, desktop using 2.08 GiB):**
+
+| micro-batch | checkpointing | peak reserved | total device used | tokens/s |
+|---|---|---|---|---|
+| 8 | off | 9.71 GiB | 11.85 GiB | 21,493 |
+| 12 | off | 11.78 GiB | 13.92 GiB | 23,487 |
+| **16** | off | **13.65 GiB** | 15.80 GiB | **23,642** |
+| 24 | off | 17.65 GiB | 19.80 GiB | 25,227 |
+| 24 | on | 8.06 GiB | 10.21 GiB | 19,156 |
+
+The finder picked 24 (largest in the 14–18 GiB band) but total device use then sits at 19.8 GiB,
+0.2 GiB under the 20 GiB "never exceed" line. Micro-batch 16 is 6% slower, gives an exact effective
+batch of 256 (24 x 11 = 264), and leaves ~8 GiB of headroom for desktop spikes. **Recommendation:
+micro-batch 16 x 16 accumulation for the long run.** Checkpointing is not needed for memory and
+costs ~25% throughput; keep it off.
+
+**Throughput / ETA.** 25,187 tok/s at B=24 (23,642 at B=16). Measured 0.2595 tokens/char on the
+packed sample -> the 408M-char corpus is ~105.8M tokens/epoch = ~807 optimizer steps at 256 x 512.
+About 1.2–1.3 h per epoch; 3 epochs ~3.75 h, 6 epochs ~7.5 h. Prediction was 2.3–3.5 h/epoch at
+12k–18k tok/s: the real machine is ~1.5x faster than the 30–45% MFU assumption (25k tok/s ≈ 63% MFU
+of the 80 TFLOPS peak).
+
+**SDPA.** Flash runs on the mask-free packed path; memory-efficient runs with the bool padding mask;
+Flash correctly refuses a mask (no silent math fallback).
+
+**Convergence.** 30 steps at B=24, LR 1e-4, 10-step warmup, real text: loss 10.58 -> 7.67
+(ln V = 10.40). The pipeline learns.
+
+**Inference latency (batch 1, 512 tokens).** GPU bf16: 10.7 ms (0.14x the instructor's 74 ms).
+CPU fp32, 16 threads: 699 ms at 512 tokens, 319 ms at 128 tokens (9.4x / 4.3x the benchmark).
+The research's 336 ms / 79 ms CPU estimate was 2–4x too optimistic; CPU inference is per-op-overhead
+bound, so truncation barely helps. **The 74 ms benchmark is met on GPU only -> the Space must be
+ZeroGPU, not CPU-basic.**
+
+**Challenges found by the reviewers before this run (fixed):** the first finder measured peak
+VRAM with a single fwd/bwd/opt step and under-reported the real accumulation peak by one fp32
+gradient copy (1.26 GiB); the first MLM-corpus decontamination used exact keys only and missed
+near-duplicate (quoted/forwarded, 419-template) variants of holdout emails; floor-division
+accumulation would have reported effective batch 256 while training at 240 for B=24;
+safetensors refuses the tied state_dict unless the alias key is dropped (helper added);
+the tied decoder leaks gradient into the [PAD] embedding row (post-accumulate hook added).
+
+## 4. 2026-10-08 00:40 — Step-4 code build (pretrain / finetune / evaluate / export / app / upload / synthetic)
+
+**What was built** (each file by one agent, then an independent adversarial reviewer who ran it, then
+a fix pass): `src/pretrain.py` (resumable MLM trainer, 1,019 lines), `src/finetune.py` (classifier
+trainer with the instructor's dials as flags; experiments.jsonl record per run), `src/evaluate.py`
+(the only script allowed to open the TREC-06 holdout; metrics, latency, plots, error analysis),
+`src/export.py` (bf16 safetensors + model card + byte-identical app/model.py), `app/app.py` +
+`requirements.txt` + `README.md` (Gradio 6 / ZeroGPU app with saliency and rule-based signals),
+`scripts/upload_to_hf.py` (dry-run by default, `--yes` required), `src/make_synthetic.py`
+(2,000 template-generated rows, 8 positive + 8 legitimate business categories, source=synthetic).
+
+**Verified by running.** pretrain: fresh run -> kill -> `--resume` reproduces the exact data order
+(window indices checked against the seeded permutation), optimizer step counter and LR; SIGINT saves
+a checkpoint and exits 0; SIGKILL between checkpoints is recovered on resume (orphan metrics rows
+dropped); a forced OOM saves and exits 3; a 5 GiB test cap trips the VRAM guard; `--max-hours` stops
+cleanly; resume refuses a changed plan or model shape. The full token cache was built as a side
+effect: 107,498,500 train tokens + 1,021,888 val tokens (0.266 tok/char), 42.8 s. finetune: resume
+reproduces batch order; `--n-layers 4` loads only the first four pretrained layers; focal(gamma=0)
+== cross-entropy; the holdout parquet is never opened (inotify-verified). 14 agents, 451 tool calls.
+
+**Bugs the reviewers caught and the fixers closed (for the challenges slide):** a `--lr` override on
+`--resume` was silently discarded by the restored optimizer state but recorded in the experiment log
+as if applied; resuming a finished fine-tune run trained one extra step at lr 0; changing frozen
+layers on resume crashed with a raw torch error; metrics CSVs kept orphan rows after a hard crash;
+evaluate/export and app each had 2–3 majors (details in docs/research/ and the agents' reports).
+
+**Changes I made myself after reading pretrain.py:** (1) the OOM handler now covers the optimizer
+step and the eval pass too, not just the forward/backward loop, so a desktop VRAM spike during the
+500-step eval cannot lose 45 minutes of work; (2) a resume with nothing left to do no longer
+overwrites run_summary.json; (3) `--resume` compares `--limit-docs` with the checkpoint before
+tokenising anything; (4) **every packed window now starts with [CLS]** (511 stream tokens follow),
+as RoBERTa prepends `<s>`: the reviewer found 0 occurrences of [CLS] in the 107M-token stream,
+which would have left the classifier's pooled position untrained until fine-tuning.
+
+**Known gaps going into the long run:** the real 337M fine-tune, evaluate and export have only run on
+tiny dev models/subsets; `--synthetic on` path untested end to end; ZeroGPU runtime behaviour
+untested until the first Space build. Dev checkpoints from the self-tests sit in
+checkpoints/{pretrain,finetune}/dev* (~15 GB) pending Nathan's OK to delete.
+
+## 5. 2026-10-08 12:28 — Pretraining run `large-24L` launched (Nathan's settings)
+
+**Decisions (Nathan, 2026-10-08).** Micro-batch 24 (17.65 GiB reserved measured; total GPU use ~20 GB
+with the desktop). 12 epochs: ~14 h at 25k tok/s, 9,552 optimizer steps, warmup 573 steps, cosine to
+10% of 1.5e-4. Rationale: dynamic masking makes every pass different and data-constrained scaling
+results show repeated passes keep helping with diminishing returns; val MLM loss (64 fixed windows
+from the held-out last 1% of documents) is evaluated every 500 steps and the run is stopped if it
+rises. Positive class is named **"Spam/Phishing"** in the app, metrics and model card (the sources do
+not separate spam from phishing). Hugging Face login confirmed (BuildITSystems).
+
+**Launch.** `EPOCHS=12 MICRO_BATCH=24 scripts/launch_pretrain.sh` -> tmux session `phish-pretrain`,
+stdout in logs/pretrain_large-24L_20261008-122800.out. Trainer: 210,368 packed windows of
+[CLS] + 511 tokens (107.7M tokens/epoch), 796 steps/epoch, micro-batch 24 x 11 accumulation =
+**effective batch 264** (256 is not divisible by 24; the realised batch is used in every number),
+checkpoints every 500 steps (keep 3, ~3.8 GB each) + final/. Token cache reused (0 s). GPU at
+launch: 20.3 GB used, 98% util, 365 W.
+
+## 6. 2026-10-08 ~13:00 — Scope change: three-way labels (legitimate / phishing / spam)
+
+**Nathan's decision.** The classifier should output legitimate, phishing, or spam. Our current positives
+cannot be split (TREC-05/07 and CEAS-08 labelled everything unwanted as "spam"), so phishing-labelled text
+from separate provenance is being added: the Champa et al. raw corpus mirror on the Hub
+(`kudzaiprichard/aura-phishing-email-corpus`, CC-BY-4.0, byte-identical to Zenodo 8339691) with
+`Nazario.csv` (1,565 phishing, 1995–2022) and `Nigerian_Fraud.csv` (3,332 advance-fee fraud), plus our
+synthetic BITS-style lures. Spam class from provenance: SpamAssassin, Enron-Spam and (if included) Ling
+spam folders. The 250k-row `locuoco/the-biggest-spam-ham-phish-email-dataset-300000` (labels ham/phish/
+spam, provenance undocumented) is inspected first and used only if its labels agree with known provenance
+and it adds rows we do not have, after deduplication against every split and the TREC-06 holdout.
+
+**Design.** One encoder, one 3-class head, partial-label loss: rows with clean provenance train with
+3-class cross-entropy; the ~70k TREC/CEAS positives train on the probability mass of {phishing, spam}
+("unwanted") so they still shape the legitimate-vs-unwanted decision. Binary roll-up P(unwanted) =
+P(phishing)+P(spam) keeps TREC-06 as the clean binary test; a new `test_holdout3` slice of clean-
+provenance rows (drawn from sources never seen by the tokenizer or the pretraining corpus) gives the
+3x3 confusion matrix. `num_labels` lives under `finetune:` in config.yaml, not under `model:`, because
+the live pretraining run refuses to resume if its model snapshot changes.
+
+## 7. 2026-10-08 13:21 — Three-way data build (build3) and the 300k-set verdict
+
+**300k set: rejected after inspection** (`docs/research/bigshp_research.md`). `locuoco/the-biggest-spam-ham-phish-
+email-dataset-300000` is 365,448 rows, not deduplicated (51.5% within-file duplicates), and is a union of
+Kaggle re-packagings of corpora we already hold: 97.6% of its ham, 92.5% of its phish and 90.2% of its
+spam rows match our text. Its ham/spam labels agree with our provenance 99.9%/99.8% because they are our
+labels copied back; its **phish label agrees only 60.5%**: 14,660 "phish" rows are SpamAssassin/Enron-Spam
+spam-folder mail, 17,714 are TREC/CEAS spam-track rows, and only 4,784 are confirmed phishing provenance
+(Nazario/Nigerian, which we take raw from the Champa mirror). **36,020 rows (9.9%) are copies of our frozen
+TREC-06 test set.** Every "phish" row went through a stopword-stripping pipeline that ham/spam rows did
+not, a preprocessing artifact that is a near-perfect label proxy and would not transfer to real inbound
+mail. Decision: not one row ingested; `data.threeway.bigshp.enabled: false` with the verdict recorded.
+
+**Champa mirror ingested** (`kudzaiprichard/aura-phishing-email-corpus`, CC-BY-4.0): Nazario 1,565 rows ->
+1,507 kept; Nigerian_Fraud 3,332 -> 2,504 kept. Drops worth knowing: 235 Nigerian-fraud rows are near-
+duplicates of TREC-06 spam (419 letters recur across corpora) and 13 match holdout keys exactly: all
+removed so the TREC-06 test stays clean; 424 more were dropped because their clusters touched both train
+and val.
+
+**Three-way splits (label3: 0 legit / 1 phishing / 2 spam / -1 unwanted-unsplit):**
+
+| split | rows | legit | phishing | spam | unwanted (TREC/CEAS positives) |
+|---|---|---|---|---|---|
+| finetune_train | 160,263 | 81,438 | 3,150 | 12,870 | 62,805 |
+| finetune_val | 15,284 | 7,002 | 340 | 763 | 7,179 |
+| test_holdout (TREC-06, unchanged rows) | 16,302 | 12,408 | - | - | 3,894 |
+| **test_holdout3 (new, clean provenance)** | 3,220 | 1,999 | 521 | 700 | - |
+
+test_holdout3 legit = CEAS-08 736 + TREC-07 1,099 + Assassin 164 (never Enron: no pretraining overlap);
+spam = Enron-Spam 628 + Assassin 72; phishing = Nazario 222 + Nigerian 299. All of it came from
+finetune_val (never seen by the tokenizer) or the new sources (never seen by anything). finetune_val
+shrank from 17,643 to 15,284 rows, so new val F1 values are not strictly comparable with the binary runs.
+Leakage: every asserted check is 0 in `check3` and the binary `check` still passes on the rewritten files.
+Existing rows in train/test_holdout are byte-identical to the binary build (data/processed/binary_snapshot).
+
+**Honest caveat.** Only 3,150 fully-labelled phishing training rows (plus 1,000 synthetic lures) carry the
+phishing-vs-spam distinction; 62,805 positives are trained as "unwanted" only. Expect the three-way split
+to be the weaker decision; the binary roll-up on TREC-06 stays the headline.
+
+## 8. 2026-10-08 13:55 — Fine-tune batching moved into torch.utils.data.DataLoader
+
+The instructor's written spec requires PyTorch's DataLoader for batching and shuffling; finetune.py had
+its own numpy sampler. Every batch now goes through `torch.utils.data.DataLoader` (`make_loader`): a
+row-index Dataset, the seeded length-bucketed batch order as `batch_sampler`, a collate function that
+pads each micro-batch to its own longest row. Verified on the CPU fixture: losses and metrics identical
+to the previous code at every step. One problem found on the way: a DataLoader draws its worker base
+seed from the global torch RNG when iterated, which shifted the dropout stream; passing an explicit
+`generator` removed the side effect (first run differed at step 1, 1.1093 vs 1.1125; identical after).
+
+## 9. 2026-10-08 14:15 — Dev checkpoints removed; fine-tuning queue armed
+
+Nathan approved both. Deleted the agents' self-test checkpoints (checkpoints/pretrain/dev,
+checkpoints/finetune/dev, checkpoints/finetune/dev-focal; 15.3 GB) and their two dev lines in
+reports/experiments.jsonl (backup in the session scratchpad). `scripts/run_experiments.sh` is running in
+tmux session `phish-finetune`, waiting for checkpoints/pretrain/large-24L/final/model.safetensors; it then
+runs ft-ce (3 epochs) and the five ablations (weighted CE, focal, synthetic on, 12 layers, scratch; 2 epochs
+each), evaluates every best/ on TREC-06 + the three-way holdout in one call, and exports ft-ce to
+model_export/ft-ce + app/. Pretraining at step 1,100 / 9,552, loss 5.96.
+
+## 10. 2026-10-08 14:30 — Presentation notebook and README
+
+`FinalProject_PhishingDetector.ipynb` (generated by `scripts/build_notebook.py`, 48 cells) mirrors the
+instructor's nine "Step N" headers verbatim and in order, builds real `torch.utils.data.DataLoader`s from the
+processed splits in Step 2, instantiates the 337M model and prints its parameter table in Step 3, plots the
+live pretraining curve in Step 4 (shows "RUN IN PROGRESS" until final/ exists), times 100 single emails
+instructor-style in Step 5, shows the app files, runs the app's predict on built-in synthetic examples,
+checks the HF login and dry-runs the upload. It executes top to bottom on the CPU in ~70 s with zero errors
+today and prints "not available yet — produced by <command>" for the 12 artifacts the training queue will
+create; re-executing it after the queue fills everything in. Reviewer catches fixed before acceptance: one
+header lacked the instructor's "(Optional)", a dev-checkpoint caption stated a wrong step count, a
+Challenges bullet misdescribed the uv torch-backend issue, and nothing kept the notebook off the GPU when
+opened from a plain shell (now forced to CPU while the pretraining run owns the GPU).
+`README.md` documents setup, every phase's commands (verified with --help), config dials, data provenance
+and licences, the leakage policy, the experiment queue, deployment, the instructor-requirement mapping and
+the limitations. Reviewer catches fixed: it had documented a fictional experiment grid and a notebook path
+that did not exist.
+
+## 11. 2026-10-08 19:15 — Hugging Face upload approved (deferred until the export exists)
+
+Nathan asked for the upload. Dry run at 19:14: the plan is a private model repo
+`BuildITSystems/phishing-detector-encoder` (model.safetensors, config.json, tokenizer.json, README.md) and a
+private ZeroGPU Space `BuildITSystems/phishing-detector-demo` (app.py, model.py, requirements.txt, README.md,
+.gitattributes + bundled weights, so no token secret is needed). It refuses to run today because
+model_export/ft-ce does not exist yet (the fine-tuning queue produces it tomorrow morning). The upload is
+scheduled to run automatically once the export lands and the dry run reports no problems. Pretraining at
+19:14: step 4,280 / 9,552 (epoch 6 of 12), train MLM loss 2.10, on schedule for ~02:00.
+
+## 12. 2026-10-09 02:45 — Pretraining status check (scheduled)
+
+Run `large-24L` at step 9,020 / 9,552 (epoch 12 of 12), ETA ~03:30: about an hour behind the 14 h
+estimate because each 500-step validation and 3.8 GB checkpoint write pauses training. Throughput
+steady at ~24.1k tokens/s, reserved VRAM 17.94 GiB, device total 20.2 GB with the desktop, GPU 86 C.
+No watchdog action, no OOM, no warnings since launch. Validation MLM loss on the held-out 1% of
+documents fell monotonically through all twelve passes (6.54 -> 6.38 -> 5.81 -> 5.25 -> 3.91 -> 3.16 ->
+2.78 -> 2.58 -> 2.44 -> 2.36 -> 2.27 -> 2.20 -> 2.14 -> 2.11 -> 2.07 -> 2.04 -> 2.03 -> 2.01), masked-token
+accuracy 14.7% -> 62.9%, so twelve passes over the same corpus did not overfit; the biggest drop came
+between steps 2,000 and 3,000 (epochs 3-4), when the model stopped predicting frequent tokens and started
+using context. Train loss 1.55 (per-step, with dropout). The fine-tuning queue is waiting for final/.
+
+## 13. 2026-10-09 03:31 — MLM pretraining run `large-24L` (complete)
+
+**Run.** `python src/pretrain.py train --run-name large-24L`; 337,476,608 parameters (EncoderForMLM, tied head). micro-batch 24 x accumulation 11 = 264 sequences x 512 tokens per step; bf16 autocast, fused AdamW [0.9, 0.98] eps 1e-06 wd 0.01 (no decay on biases/LayerNorm), clip 1.0; cosine schedule peak 1.5e-04 -> 10% over 9552 planned steps (12 epochs x 796 steps) with 573 warmup steps; dynamic 15% masking 80/10/10, loss on masked positions only; 210,368 packed 512-token windows (226,273 docs), 1,999 val windows.
+
+**Result.** 9552 optimizer steps (12.00 epochs), 1291.1M tokens seen, total training time 15h 03m 05s (54,186 s) over 1 invocation(s), mean 23,889 tokens/s; peak VRAM reserved 17.94 GiB / allocated 17.49 GiB; final train MLM loss 1.5237, val MLM loss 1.9863 (masked-token accuracy 62.9%).
+
+**Artifacts.** checkpoints/pretrain/large-24L/ (metrics.csv, run_summary.json, step-*/ and final/); log /home/nathank/BITS/dev/Phishing Model/logs/pretrain-20261008-122802.log.
+
+## 14. 2026-10-09 04:07 — Hand-off to fine-tuning (scheduled check)
+
+Pretraining `large-24L` finished at 03:31 after 15 h 03 m: 9,552 optimizer steps, 1.29 B tokens seen
+(12 passes over 107.5 M), mean 23,889 tokens/s, peak VRAM reserved 17.94 GiB, final train MLM loss 1.52,
+validation MLM loss 1.99 with 62.9% masked-token accuracy (from 10.4 / ~0% at random init). final/ holds
+the fp32 encoder (1.35 GB) + tokenizer + config. The watchdog saw final/ and exited without ever acting.
+The queue started `ft-ce` at 03:36: 3 epochs x 5,009 steps at batch 32 (micro-batch 16), 13.1 GiB reserved,
+~75 examples/s, ETA ~05:30 for this run. First validation at step 3,750 (0.75 epoch), 15,284 rows:
+unwanted-vs-legitimate F1 0.9855, ROC-AUC 0.9995, PR-AUC 0.9996;
+three-way slice (n=8105) macro-F1 0.9775, accuracy 0.9921, per-class F1 [legit, phishing, spam] = [0.996 0.967 0.970];
+phishing-vs-spam on true unwanted rows [pp ps sp ss] = [340 0 6 757]. Val F1 is not comparable with the earlier binary dev numbers (val shrank by the 2,699 rows moved to test_holdout3).
+Queue estimate: ft-ce done ~05:30, then five 2-epoch ablations (~1 h each + evaluation) -> ~11:00-11:30.
+
+## Fine-tune run `ft-ce` — 2026-10-09 04:28
+**Dials.** init pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final; 3 classes; 24 layers, 0 frozen; loss cross_entropy; lr 3e-05; batch 32 (micro 16); epochs 3 planned; synthetic off (0 rows); selection val_f1_unwanted; train rows 160,263, val rows 15,284; params 337,444,867 (337,444,867 trainable).
+**Result.** 6250 steps, stopped by early_stopping; training time 51.5 min (eval 4.8 min of it); peak reserved 13.13 GiB. Best val at step 5000 (epoch 0), Unwanted (phishing or spam) roll-up: F1 0.9929, precision 0.9938, recall 0.9920, accuracy 0.9923, ROC-AUC 0.9996, PR-AUC 0.9996; F1-optimal threshold 0.318. Three-way on 8,105 fully-labelled val rows: macro-F1 0.9782, accuracy 0.9922; per-class F1 Legitimate 0.9958, Phishing 0.9672, Spam 0.9716; phishing-vs-spam on true unwanted (n=1,103): phish->phish 340, phish->spam 0, spam->phish 4, spam->spam 759. Checkpoints: /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-ce/best, /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-ce/last.
+
+## 15. 2026-10-09 04:28 — Evaluation on the test + test3 splits (evaluate.py run, `ft-ce`)
+
+**Checkpoint.** `/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-ce/best` (3-class head; three-way label contract of RUN_LOG entry 6). Binary roll-up metrics use the same definitions as the binary-era runs so they stay comparable.
+
+### test split (TREC-06 binary holdout): `test_holdout.parquet`, 16,302 rows (3,894 unwanted, 23.9%; label3 legit 12,408, unwanted 3,894). Inference 285.1 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9404** (232 missed of 3,894) |
+| precision | 0.8610 (591 false alarms) |
+| F1 | 0.8990 |
+| accuracy | 0.9495 |
+| specificity | 0.9524 |
+| ROC-AUC | 0.9875 |
+| PR-AUC | 0.9626 |
+| recall @ 1% FPR | 0.7745 |
+| recall @ 0.1% FPR | 0.3529 |
+| best-F1 threshold | 0.821 (F1 0.9090) |
+| credential-lure slice | 398 rows (61.1% labelled positive), recall 0.9959, precision 0.8996 |
+
+**Three-way:** no phishing-vs-spam ground truth on this split (no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)); informational: of the 3,662 flagged true positives the model called 1,294 phishing and 2,368 spam.
+
+### test3 split (three-way holdout): `test_holdout3.parquet`, 3,220 rows (1,221 unwanted, 37.9%; label3 legit 1,999, phish 521, spam 700). Inference 240.9 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9853** (18 missed of 1,221) |
+| precision | 0.9877 (15 false alarms) |
+| F1 | 0.9865 |
+| accuracy | 0.9898 |
+| specificity | 0.9925 |
+| ROC-AUC | 0.9992 |
+| PR-AUC | 0.9985 |
+| recall @ 1% FPR | 0.9869 |
+| recall @ 0.1% FPR | 0.8690 |
+| best-F1 threshold | 0.411 (F1 0.9869) |
+| credential-lure slice | 275 rows (97.5% labelled positive), recall 0.9925, precision 1.0000 |
+
+**Three-way** on the 3,220 rows with label3 in {0,1,2} (0 unsplit rows enter the roll-up only; rule: rollup_then_argmax): accuracy 0.9863, **macro-F1 0.9822**; Legitimate P 0.991 / R 0.992 / F1 0.992 (n=1,999); Phishing P 0.981 / R 0.973 / F1 0.977 (n=521); Spam P 0.977 / R 0.979 / F1 0.978 (n=700). 3x3 (rows true, cols predicted Legitimate/Phishing/Spam): Legitimate 1,984/4/11; Phishing 9/507/5; Spam 9/6/685. Phishing-vs-spam on the 1,221 true unwanted rows: kind accuracy 0.9894 (7 phishing->spam, 6 spam->phishing).
+
+| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |
+|---|---:|---:|---:|---:|---:|---:|
+| Assassin | 236 | 164 | 0 | 72 | 0.9449 | 0.9464 |
+| CEAS-08 | 736 | 736 | 0 | 0 | 0.9905 | 0.9952 |
+| Enron | 628 | 0 | 0 | 628 | 0.9889 | 0.9944 |
+| Nazario | 222 | 0 | 222 | 0 | 0.9505 | 0.9746 |
+| Nigerian_Fraud | 299 | 0 | 299 | 0 | 0.9900 | 0.9950 |
+| TREC-07 | 1,099 | 1,099 | 0 | 0 | 0.9973 | 0.9986 |
+
+**Latency (batch 1, tokenize + forward + softmax, timeit.default_timer) vs the instructor's 74 ms.** GPU bf16: 9.95 ms avg / 8.72 ms p50 over 100 emails (0.13x). CPU fp32 (16 threads): 514.07 ms avg / 526.19 ms p50 over 100 emails (6.95x).
+
+**Length confound (test).** FN mean 256 tokens vs TP 287; FP 763 vs TN 848 (see error_analysis.md for the per-bucket table).
+
+**Artifacts.** `reports/eval_ft-ce`: metrics.json, latency.json, confusion_matrix.png, roc_curve.png, pr_curve.png, error_analysis.md; metrics_test3.json, latency_test3.json, confusion_matrix_test3.png, confusion_matrix3_test3.png, roc_curve_test3.png, pr_curve_test3.png, error_analysis_test3.md (training curves: reports/pretrain_curves.png and finetune_curves.png from pretrain.py / finetune.py `plot`).
+
+## Fine-tune run `ft-weighted` — 2026-10-09 05:55
+**Dials.** init pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final; 3 classes; 24 layers, 0 frozen; loss weighted_cross_entropy; lr 3e-05; batch 32 (micro 16); epochs 2 planned; synthetic off (0 rows); selection val_f1_unwanted; train rows 160,263, val rows 15,284; params 337,444,867 (337,444,867 trainable).
+**Result.** 10018 steps, stopped by epochs_done; training time 83.6 min (eval 8.0 min of it); peak reserved 13.13 GiB. Best val at step 10000 (epoch 1), Unwanted (phishing or spam) roll-up: F1 0.9937, precision 0.9911, recall 0.9964, accuracy 0.9932, ROC-AUC 0.9998, PR-AUC 0.9998; F1-optimal threshold 0.899. Three-way on 8,105 fully-labelled val rows: macro-F1 0.9775, accuracy 0.9903; per-class F1 Legitimate 0.9944, Phishing 0.9798, Spam 0.9583; phishing-vs-spam on true unwanted (n=1,103): phish->phish 339, phish->spam 1, spam->phish 2, spam->spam 761. Checkpoints: /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-weighted/best, /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-weighted/last.
+
+## 16. 2026-10-09 05:55 — Evaluation on the test + test3 splits (evaluate.py run, `ft-weighted`)
+
+**Checkpoint.** `/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-weighted/best` (3-class head; three-way label contract of RUN_LOG entry 6). Binary roll-up metrics use the same definitions as the binary-era runs so they stay comparable.
+
+### test split (TREC-06 binary holdout): `test_holdout.parquet`, 16,302 rows (3,894 unwanted, 23.9%; label3 legit 12,408, unwanted 3,894). Inference 283.4 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9818** (71 missed of 3,894) |
+| precision | 0.8307 (779 false alarms) |
+| F1 | 0.9000 |
+| accuracy | 0.9479 |
+| specificity | 0.9372 |
+| ROC-AUC | 0.9943 |
+| PR-AUC | 0.9833 |
+| recall @ 1% FPR | 0.8767 |
+| recall @ 0.1% FPR | 0.6335 |
+| best-F1 threshold | 0.989 (F1 0.9383) |
+| credential-lure slice | 398 rows (61.1% labelled positive), recall 0.9959, precision 0.9565 |
+
+**Three-way:** no phishing-vs-spam ground truth on this split (no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)); informational: of the 3,823 flagged true positives the model called 801 phishing and 3,022 spam.
+
+### test3 split (three-way holdout): `test_holdout3.parquet`, 3,220 rows (1,221 unwanted, 37.9%; label3 legit 1,999, phish 521, spam 700). Inference 238.3 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9934** (8 missed of 1,221) |
+| precision | 0.9790 (26 false alarms) |
+| F1 | 0.9862 |
+| accuracy | 0.9894 |
+| specificity | 0.9870 |
+| ROC-AUC | 0.9993 |
+| PR-AUC | 0.9988 |
+| recall @ 1% FPR | 0.9910 |
+| recall @ 0.1% FPR | 0.7060 |
+| best-F1 threshold | 0.728 (F1 0.9878) |
+| credential-lure slice | 275 rows (97.5% labelled positive), recall 1.0000, precision 1.0000 |
+
+**Three-way** on the 3,220 rows with label3 in {0,1,2} (0 unsplit rows enter the roll-up only; rule: rollup_then_argmax): accuracy 0.9876, **macro-F1 0.9852**; Legitimate P 0.996 / R 0.987 / F1 0.991 (n=1,999); Phishing P 0.988 / R 0.987 / F1 0.988 (n=521); Spam P 0.964 / R 0.990 / F1 0.977 (n=700). 3x3 (rows true, cols predicted Legitimate/Phishing/Spam): Legitimate 1,973/5/21; Phishing 2/514/5; Spam 6/1/693. Phishing-vs-spam on the 1,221 true unwanted rows: kind accuracy 0.9943 (6 phishing->spam, 1 spam->phishing).
+
+| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |
+|---|---:|---:|---:|---:|---:|---:|
+| Assassin | 236 | 164 | 0 | 72 | 0.9619 | 0.9603 |
+| CEAS-08 | 736 | 736 | 0 | 0 | 0.9823 | 0.9911 |
+| Enron | 628 | 0 | 0 | 628 | 0.9936 | 0.9968 |
+| Nazario | 222 | 0 | 222 | 0 | 0.9820 | 0.9909 |
+| Nigerian_Fraud | 299 | 0 | 299 | 0 | 0.9900 | 0.9950 |
+| TREC-07 | 1,099 | 1,099 | 0 | 0 | 0.9936 | 0.9968 |
+
+**Latency (batch 1, tokenize + forward + softmax, timeit.default_timer) vs the instructor's 74 ms.** GPU bf16: 10.12 ms avg / 8.95 ms p50 over 100 emails (0.14x). CPU fp32 (16 threads): 411.57 ms avg / 397.57 ms p50 over 100 emails (5.56x).
+
+**Length confound (test).** FN mean 310 tokens vs TP 284; FP 1779 vs TN 782 (see error_analysis.md for the per-bucket table).
+
+**Artifacts.** `reports/eval_ft-weighted`: metrics.json, latency.json, confusion_matrix.png, roc_curve.png, pr_curve.png, error_analysis.md; metrics_test3.json, latency_test3.json, confusion_matrix_test3.png, confusion_matrix3_test3.png, roc_curve_test3.png, pr_curve_test3.png, error_analysis_test3.md (training curves: reports/pretrain_curves.png and finetune_curves.png from pretrain.py / finetune.py `plot`).
+
+## Fine-tune run `ft-focal` — 2026-10-09 07:21
+**Dials.** init pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final; 3 classes; 24 layers, 0 frozen; loss focal (gamma 2.0); lr 3e-05; batch 32 (micro 16); epochs 2 planned; synthetic off (0 rows); selection val_f1_unwanted; train rows 160,263, val rows 15,284; params 337,444,867 (337,444,867 trainable).
+**Result.** 10018 steps, stopped by early_stopping; training time 83.9 min (eval 8.0 min of it); peak reserved 13.13 GiB. Best val at step 8750 (epoch 1), Unwanted (phishing or spam) roll-up: F1 0.9951, precision 0.9954, recall 0.9948, accuracy 0.9947, ROC-AUC 0.9998, PR-AUC 0.9998; F1-optimal threshold 0.421. Three-way on 8,105 fully-labelled val rows: macro-F1 0.9869, accuracy 0.9947; per-class F1 Legitimate 0.9971, Phishing 0.9868, Spam 0.9768; phishing-vs-spam on true unwanted (n=1,103): phish->phish 339, phish->spam 1, spam->phish 1, spam->spam 762. Checkpoints: /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best, /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/last.
+
+## 17. 2026-10-09 07:22 — Evaluation on the test + test3 splits (evaluate.py run, `ft-focal`)
+
+**Checkpoint.** `/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best` (3-class head; three-way label contract of RUN_LOG entry 6). Binary roll-up metrics use the same definitions as the binary-era runs so they stay comparable.
+
+### test split (TREC-06 binary holdout): `test_holdout.parquet`, 16,302 rows (3,894 unwanted, 23.9%; label3 legit 12,408, unwanted 3,894). Inference 282.8 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9741** (101 missed of 3,894) |
+| precision | 0.8794 (520 false alarms) |
+| F1 | 0.9243 |
+| accuracy | 0.9619 |
+| specificity | 0.9581 |
+| ROC-AUC | 0.9946 |
+| PR-AUC | 0.9848 |
+| recall @ 1% FPR | 0.8955 |
+| recall @ 0.1% FPR | 0.6497 |
+| best-F1 threshold | 0.682 (F1 0.9397) |
+| credential-lure slice | 398 rows (61.1% labelled positive), recall 0.9959, precision 0.9758 |
+
+**Three-way:** no phishing-vs-spam ground truth on this split (no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)); informational: of the 3,793 flagged true positives the model called 974 phishing and 2,819 spam.
+
+### test3 split (three-way holdout): `test_holdout3.parquet`, 3,220 rows (1,221 unwanted, 37.9%; label3 legit 1,999, phish 521, spam 700). Inference 238.7 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9836** (20 missed of 1,221) |
+| precision | 0.9893 (13 false alarms) |
+| F1 | 0.9864 |
+| accuracy | 0.9898 |
+| specificity | 0.9935 |
+| ROC-AUC | 0.9993 |
+| PR-AUC | 0.9989 |
+| recall @ 1% FPR | 0.9869 |
+| recall @ 0.1% FPR | 0.8788 |
+| best-F1 threshold | 0.475 (F1 0.9877) |
+| credential-lure slice | 275 rows (97.5% labelled positive), recall 1.0000, precision 1.0000 |
+
+**Three-way** on the 3,220 rows with label3 in {0,1,2} (0 unsplit rows enter the roll-up only; rule: rollup_then_argmax): accuracy 0.9885, **macro-F1 0.9867**; Legitimate P 0.990 / R 0.993 / F1 0.992 (n=1,999); Phishing P 0.996 / R 0.987 / F1 0.991 (n=521); Spam P 0.979 / R 0.976 / F1 0.977 (n=700). 3x3 (rows true, cols predicted Legitimate/Phishing/Spam): Legitimate 1,986/1/12; Phishing 4/514/3; Spam 16/1/683. Phishing-vs-spam on the 1,221 true unwanted rows: kind accuracy 0.9959 (4 phishing->spam, 1 spam->phishing).
+
+| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |
+|---|---:|---:|---:|---:|---:|---:|
+| Assassin | 236 | 164 | 0 | 72 | 0.9534 | 0.9458 |
+| CEAS-08 | 736 | 736 | 0 | 0 | 0.9905 | 0.9952 |
+| Enron | 628 | 0 | 0 | 628 | 0.9841 | 0.9920 |
+| Nazario | 222 | 0 | 222 | 0 | 0.9865 | 0.9932 |
+| Nigerian_Fraud | 299 | 0 | 299 | 0 | 0.9866 | 0.9933 |
+| TREC-07 | 1,099 | 1,099 | 0 | 0 | 0.9982 | 0.9991 |
+
+**Latency (batch 1, tokenize + forward + softmax, timeit.default_timer) vs the instructor's 74 ms.** GPU bf16: 10.34 ms avg / 9.01 ms p50 over 100 emails (0.14x). CPU fp32 (16 threads): 357.98 ms avg / 340.34 ms p50 over 100 emails (4.84x).
+
+**Length confound (test).** FN mean 322 tokens vs TP 284; FP 1237 vs TN 827 (see error_analysis.md for the per-bucket table).
+
+**Artifacts.** `reports/eval_ft-focal`: metrics.json, latency.json, confusion_matrix.png, roc_curve.png, pr_curve.png, error_analysis.md; metrics_test3.json, latency_test3.json, confusion_matrix_test3.png, confusion_matrix3_test3.png, roc_curve_test3.png, pr_curve_test3.png, error_analysis_test3.md (training curves: reports/pretrain_curves.png and finetune_curves.png from pretrain.py / finetune.py `plot`).
+
+## Fine-tune run `ft-synthetic` — 2026-10-09 08:16
+**Dials.** init pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final; 3 classes; 24 layers, 0 frozen; loss cross_entropy; lr 3e-05; batch 32 (micro 16); epochs 2 planned; synthetic on (2000 rows); selection val_f1_unwanted; train rows 162,263, val rows 15,284; params 337,444,867 (337,444,867 trainable).
+**Result.** 6250 steps, stopped by early_stopping; training time 52.0 min (eval 4.8 min of it); peak reserved 13.13 GiB. Best val at step 5000 (epoch 0), Unwanted (phishing or spam) roll-up: F1 0.9927, precision 0.9904, recall 0.9949, accuracy 0.9920, ROC-AUC 0.9996, PR-AUC 0.9997; F1-optimal threshold 0.609. Three-way on 8,105 fully-labelled val rows: macro-F1 0.9665, accuracy 0.9891; per-class F1 Legitimate 0.9940, Phishing 0.9352, Spam 0.9704; phishing-vs-spam on true unwanted (n=1,103): phish->phish 340, phish->spam 0, spam->phish 6, spam->spam 757. Checkpoints: /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-synthetic/best, /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-synthetic/last.
+
+## 18. 2026-10-09 08:16 — Evaluation on the test + test3 splits (evaluate.py run, `ft-synthetic`)
+
+**Checkpoint.** `/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-synthetic/best` (3-class head; three-way label contract of RUN_LOG entry 6). Binary roll-up metrics use the same definitions as the binary-era runs so they stay comparable.
+
+### test split (TREC-06 binary holdout): `test_holdout.parquet`, 16,302 rows (3,894 unwanted, 23.9%; label3 legit 12,408, unwanted 3,894). Inference 282.3 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9679** (125 missed of 3,894) |
+| precision | 0.8402 (717 false alarms) |
+| F1 | 0.8995 |
+| accuracy | 0.9483 |
+| specificity | 0.9422 |
+| ROC-AUC | 0.9908 |
+| PR-AUC | 0.9716 |
+| recall @ 1% FPR | 0.7871 |
+| recall @ 0.1% FPR | 0.4779 |
+| best-F1 threshold | 0.962 (F1 0.9186) |
+| credential-lure slice | 398 rows (61.1% labelled positive), recall 0.9959, precision 0.9528 |
+
+**Three-way:** no phishing-vs-spam ground truth on this split (no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)); informational: of the 3,769 flagged true positives the model called 1,486 phishing and 2,283 spam.
+
+### test3 split (three-way holdout): `test_holdout3.parquet`, 3,220 rows (1,221 unwanted, 37.9%; label3 legit 1,999, phish 521, spam 700). Inference 238.3 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9885** (14 missed of 1,221) |
+| precision | 0.9861 (17 false alarms) |
+| F1 | 0.9873 |
+| accuracy | 0.9904 |
+| specificity | 0.9915 |
+| ROC-AUC | 0.9993 |
+| PR-AUC | 0.9988 |
+| recall @ 1% FPR | 0.9885 |
+| recall @ 0.1% FPR | 0.8395 |
+| best-F1 threshold | 0.508 (F1 0.9873) |
+| credential-lure slice | 275 rows (97.5% labelled positive), recall 1.0000, precision 1.0000 |
+
+**Three-way** on the 3,220 rows with label3 in {0,1,2} (0 unsplit rows enter the roll-up only; rule: rollup_then_argmax): accuracy 0.9860, **macro-F1 0.9814**; Legitimate P 0.993 / R 0.991 / F1 0.992 (n=1,999); Phishing P 0.968 / R 0.987 / F1 0.977 (n=521); Spam P 0.980 / R 0.970 / F1 0.975 (n=700). 3x3 (rows true, cols predicted Legitimate/Phishing/Spam): Legitimate 1,982/6/11; Phishing 4/514/3; Spam 10/11/679. Phishing-vs-spam on the 1,221 true unwanted rows: kind accuracy 0.9869 (5 phishing->spam, 11 spam->phishing).
+
+| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |
+|---|---:|---:|---:|---:|---:|---:|
+| Assassin | 236 | 164 | 0 | 72 | 0.9280 | 0.9296 |
+| CEAS-08 | 736 | 736 | 0 | 0 | 0.9851 | 0.9925 |
+| Enron | 628 | 0 | 0 | 628 | 0.9873 | 0.9936 |
+| Nazario | 222 | 0 | 222 | 0 | 0.9820 | 0.9909 |
+| Nigerian_Fraud | 299 | 0 | 299 | 0 | 0.9900 | 0.9950 |
+| TREC-07 | 1,099 | 1,099 | 0 | 0 | 0.9982 | 0.9991 |
+
+**Latency (batch 1, tokenize + forward + softmax, timeit.default_timer) vs the instructor's 74 ms.** GPU bf16: 10.05 ms avg / 8.9 ms p50 over 100 emails (0.14x). CPU fp32 (16 threads): 481.35 ms avg / 464.28 ms p50 over 100 emails (6.50x).
+
+**Length confound (test).** FN mean 256 tokens vs TP 286; FP 686 vs TN 854 (see error_analysis.md for the per-bucket table).
+
+**Artifacts.** `reports/eval_ft-synthetic`: metrics.json, latency.json, confusion_matrix.png, roc_curve.png, pr_curve.png, error_analysis.md; metrics_test3.json, latency_test3.json, confusion_matrix_test3.png, confusion_matrix3_test3.png, roc_curve_test3.png, pr_curve_test3.png, error_analysis_test3.md (training curves: reports/pretrain_curves.png and finetune_curves.png from pretrain.py / finetune.py `plot`).
+
+## Fine-tune run `ft-12L` — 2026-10-09 08:54
+**Dials.** init pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final; 3 classes; 12 layers, 0 frozen; loss cross_entropy; lr 3e-05; batch 32 (micro 16); epochs 2 planned; synthetic off (0 rows); selection val_f1_unwanted; train rows 160,263, val rows 15,284; params 186,290,179 (186,290,179 trainable).
+**Result.** 8750 steps, stopped by early_stopping; training time 35.3 min (eval 3.1 min of it); peak reserved 6.97 GiB. Best val at step 6250 (epoch 1), Unwanted (phishing or spam) roll-up: F1 0.9925, precision 0.9936, recall 0.9914, accuracy 0.9919, ROC-AUC 0.9995, PR-AUC 0.9996; F1-optimal threshold 0.171. Three-way on 8,105 fully-labelled val rows: macro-F1 0.9774, accuracy 0.9916; per-class F1 Legitimate 0.9954, Phishing 0.9695, Spam 0.9673; phishing-vs-spam on true unwanted (n=1,103): phish->phish 338, phish->spam 2, spam->phish 2, spam->spam 761. Checkpoints: /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-12L/best, /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-12L/last.
+
+## 19. 2026-10-09 08:54 — Evaluation on the test + test3 splits (evaluate.py run, `ft-12L`)
+
+**Checkpoint.** `/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-12L/best` (3-class head; three-way label contract of RUN_LOG entry 6). Binary roll-up metrics use the same definitions as the binary-era runs so they stay comparable.
+
+### test split (TREC-06 binary holdout): `test_holdout.parquet`, 16,302 rows (3,894 unwanted, 23.9%; label3 legit 12,408, unwanted 3,894). Inference 591.7 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 1.7 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9461** (210 missed of 3,894) |
+| precision | 0.8120 (853 false alarms) |
+| F1 | 0.8739 |
+| accuracy | 0.9348 |
+| specificity | 0.9313 |
+| ROC-AUC | 0.9836 |
+| PR-AUC | 0.9513 |
+| recall @ 1% FPR | 0.6608 |
+| recall @ 0.1% FPR | 0.4073 |
+| best-F1 threshold | 0.955 (F1 0.8852) |
+| credential-lure slice | 398 rows (61.1% labelled positive), recall 0.9959, precision 0.9030 |
+
+**Three-way:** no phishing-vs-spam ground truth on this split (no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)); informational: of the 3,684 flagged true positives the model called 802 phishing and 2,882 spam.
+
+### test3 split (three-way holdout): `test_holdout3.parquet`, 3,220 rows (1,221 unwanted, 37.9%; label3 legit 1,999, phish 521, spam 700). Inference 501.6 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 1.7 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9787** (26 missed of 1,221) |
+| precision | 0.9860 (17 false alarms) |
+| F1 | 0.9823 |
+| accuracy | 0.9866 |
+| specificity | 0.9915 |
+| ROC-AUC | 0.9991 |
+| PR-AUC | 0.9983 |
+| recall @ 1% FPR | 0.9795 |
+| recall @ 0.1% FPR | 0.7748 |
+| best-F1 threshold | 0.169 (F1 0.9849) |
+| credential-lure slice | 275 rows (97.5% labelled positive), recall 0.9888, precision 1.0000 |
+
+**Three-way** on the 3,220 rows with label3 in {0,1,2} (0 unsplit rows enter the roll-up only; rule: rollup_then_argmax): accuracy 0.9842, **macro-F1 0.9807**; Legitimate P 0.987 / R 0.991 / F1 0.989 (n=1,999); Phishing P 0.988 / R 0.973 / F1 0.981 (n=521); Spam P 0.973 / R 0.971 / F1 0.972 (n=700). 3x3 (rows true, cols predicted Legitimate/Phishing/Spam): Legitimate 1,982/5/12; Phishing 7/507/7; Spam 19/1/680. Phishing-vs-spam on the 1,221 true unwanted rows: kind accuracy 0.9910 (10 phishing->spam, 1 spam->phishing).
+
+| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |
+|---|---:|---:|---:|---:|---:|---:|
+| Assassin | 236 | 164 | 0 | 72 | 0.9619 | 0.9556 |
+| CEAS-08 | 736 | 736 | 0 | 0 | 0.9864 | 0.9932 |
+| Enron | 628 | 0 | 0 | 628 | 0.9793 | 0.9895 |
+| Nazario | 222 | 0 | 222 | 0 | 0.9685 | 0.9840 |
+| Nigerian_Fraud | 299 | 0 | 299 | 0 | 0.9766 | 0.9882 |
+| TREC-07 | 1,099 | 1,099 | 0 | 0 | 0.9955 | 0.9977 |
+
+**Latency (batch 1, tokenize + forward + softmax, timeit.default_timer) vs the instructor's 74 ms.** GPU bf16: 5.05 ms avg / 4.42 ms p50 over 100 emails (0.07x). CPU fp32 (16 threads): 246.62 ms avg / 242.67 ms p50 over 100 emails (3.33x).
+
+**Length confound (test).** FN mean 260 tokens vs TP 286; FP 1105 vs TN 825 (see error_analysis.md for the per-bucket table).
+
+**Artifacts.** `reports/eval_ft-12L`: metrics.json, latency.json, confusion_matrix.png, roc_curve.png, pr_curve.png, error_analysis.md; metrics_test3.json, latency_test3.json, confusion_matrix_test3.png, confusion_matrix3_test3.png, roc_curve_test3.png, pr_curve_test3.png, error_analysis_test3.md (training curves: reports/pretrain_curves.png and finetune_curves.png from pretrain.py / finetune.py `plot`).
+
+## Fine-tune run `ft-scratch` — 2026-10-09 09:57
+**Dials.** init scratch; 3 classes; 24 layers, 0 frozen; loss cross_entropy; lr 3e-05; batch 32 (micro 16); epochs 2 planned; synthetic off (0 rows); selection val_f1_unwanted; train rows 160,263, val rows 15,284; params 337,444,867 (337,444,867 trainable).
+**Result.** 7500 steps, stopped by early_stopping; training time 60.9 min (eval 5.5 min of it); peak reserved 13.13 GiB. Best val at step 5009 (epoch 0), Unwanted (phishing or spam) roll-up: F1 0.9875, precision 0.9864, recall 0.9885, accuracy 0.9864, ROC-AUC 0.9985, PR-AUC 0.9986; F1-optimal threshold 0.601. Three-way on 8,105 fully-labelled val rows: macro-F1 0.9538, accuracy 0.9832; per-class F1 Legitimate 0.9915, Phishing 0.9398, Spam 0.9300; phishing-vs-spam on true unwanted (n=1,103): phish->phish 322, phish->spam 18, spam->phish 1, spam->spam 762. Checkpoints: /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-scratch/best, /home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-scratch/last.
+
+## 20. 2026-10-09 09:57 — Evaluation on the test + test3 splits (evaluate.py run, `ft-scratch`)
+
+**Checkpoint.** `/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-scratch/best` (3-class head; three-way label contract of RUN_LOG entry 6). Binary roll-up metrics use the same definitions as the binary-era runs so they stay comparable.
+
+### test split (TREC-06 binary holdout): `test_holdout.parquet`, 16,302 rows (3,894 unwanted, 23.9%; label3 legit 12,408, unwanted 3,894). Inference 282.4 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9165** (325 missed of 3,894) |
+| precision | 0.7752 (1,035 false alarms) |
+| F1 | 0.8400 |
+| accuracy | 0.9166 |
+| specificity | 0.9166 |
+| ROC-AUC | 0.9755 |
+| PR-AUC | 0.9456 |
+| recall @ 1% FPR | 0.7773 |
+| recall @ 0.1% FPR | 0.4954 |
+| best-F1 threshold | 0.941 (F1 0.8772) |
+| credential-lure slice | 398 rows (61.1% labelled positive), recall 0.9959, precision 0.9308 |
+
+**Three-way:** no phishing-vs-spam ground truth on this split (no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)); informational: of the 3,569 flagged true positives the model called 332 phishing and 3,237 spam.
+
+### test3 split (three-way holdout): `test_holdout3.parquet`, 3,220 rows (1,221 unwanted, 37.9%; label3 legit 1,999, phish 521, spam 700). Inference 237.5 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9771** (28 missed of 1,221) |
+| precision | 0.9763 (29 false alarms) |
+| F1 | 0.9767 |
+| accuracy | 0.9823 |
+| specificity | 0.9855 |
+| ROC-AUC | 0.9983 |
+| PR-AUC | 0.9974 |
+| recall @ 1% FPR | 0.9672 |
+| recall @ 0.1% FPR | 0.8354 |
+| best-F1 threshold | 0.366 (F1 0.9776) |
+| credential-lure slice | 275 rows (97.5% labelled positive), recall 0.9925, precision 1.0000 |
+
+**Three-way** on the 3,220 rows with label3 in {0,1,2} (0 unsplit rows enter the roll-up only; rule: rollup_then_argmax): accuracy 0.9727, **macro-F1 0.9634**; Legitimate P 0.986 / R 0.985 / F1 0.986 (n=1,999); Phishing P 0.990 / R 0.927 / F1 0.957 (n=521); Spam P 0.925 / R 0.970 / F1 0.947 (n=700). 3x3 (rows true, cols predicted Legitimate/Phishing/Spam): Legitimate 1,970/3/26; Phishing 9/483/29; Spam 19/2/679. Phishing-vs-spam on the 1,221 true unwanted rows: kind accuracy 0.9713 (32 phishing->spam, 3 spam->phishing).
+
+| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |
+|---|---:|---:|---:|---:|---:|---:|
+| Assassin | 236 | 164 | 0 | 72 | 0.9407 | 0.9330 |
+| CEAS-08 | 736 | 736 | 0 | 0 | 0.9796 | 0.9897 |
+| Enron | 628 | 0 | 0 | 628 | 0.9777 | 0.9887 |
+| Nazario | 222 | 0 | 222 | 0 | 0.8964 | 0.9454 |
+| Nigerian_Fraud | 299 | 0 | 299 | 0 | 0.9498 | 0.9743 |
+| TREC-07 | 1,099 | 1,099 | 0 | 0 | 0.9936 | 0.9968 |
+
+**Latency (batch 1, tokenize + forward + softmax, timeit.default_timer) vs the instructor's 74 ms.** GPU bf16: 10.06 ms avg / 8.92 ms p50 over 100 emails (0.14x). CPU fp32 (16 threads): 354.05 ms avg / 338.14 ms p50 over 100 emails (4.78x).
+
+**Length confound (test).** FN mean 334 tokens vs TP 280; FP 728 vs TN 855 (see error_analysis.md for the per-bucket table).
+
+**Artifacts.** `reports/eval_ft-scratch`: metrics.json, latency.json, confusion_matrix.png, roc_curve.png, pr_curve.png, error_analysis.md; metrics_test3.json, latency_test3.json, confusion_matrix_test3.png, confusion_matrix3_test3.png, roc_curve_test3.png, pr_curve_test3.png, error_analysis_test3.md (training curves: reports/pretrain_curves.png and finetune_curves.png from pretrain.py / finetune.py `plot`).
+
+## 21. 2026-10-09 09:59 — Evaluation on the test + test3 splits (evaluate.py run, `ft-ce`)
+
+**Checkpoint.** `/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-ce/best` (3-class head; three-way label contract of RUN_LOG entry 6). Binary roll-up metrics use the same definitions as the binary-era runs so they stay comparable.
+
+### test split (TREC-06 binary holdout): `test_holdout.parquet`, 16,302 rows (3,894 unwanted, 23.9%; label3 legit 12,408, unwanted 3,894). Inference 285.2 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9404** (232 missed of 3,894) |
+| precision | 0.8610 (591 false alarms) |
+| F1 | 0.8990 |
+| accuracy | 0.9495 |
+| specificity | 0.9524 |
+| ROC-AUC | 0.9875 |
+| PR-AUC | 0.9626 |
+| recall @ 1% FPR | 0.7745 |
+| recall @ 0.1% FPR | 0.3529 |
+| best-F1 threshold | 0.821 (F1 0.9090) |
+| credential-lure slice | 398 rows (61.1% labelled positive), recall 0.9959, precision 0.8996 |
+
+**Three-way:** no phishing-vs-spam ground truth on this split (no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)); informational: of the 3,662 flagged true positives the model called 1,294 phishing and 2,368 spam.
+
+### test3 split (three-way holdout): `test_holdout3.parquet`, 3,220 rows (1,221 unwanted, 37.9%; label3 legit 1,999, phish 521, spam 700). Inference 239.6 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9853** (18 missed of 1,221) |
+| precision | 0.9877 (15 false alarms) |
+| F1 | 0.9865 |
+| accuracy | 0.9898 |
+| specificity | 0.9925 |
+| ROC-AUC | 0.9992 |
+| PR-AUC | 0.9985 |
+| recall @ 1% FPR | 0.9869 |
+| recall @ 0.1% FPR | 0.8690 |
+| best-F1 threshold | 0.411 (F1 0.9869) |
+| credential-lure slice | 275 rows (97.5% labelled positive), recall 0.9925, precision 1.0000 |
+
+**Three-way** on the 3,220 rows with label3 in {0,1,2} (0 unsplit rows enter the roll-up only; rule: rollup_then_argmax): accuracy 0.9863, **macro-F1 0.9822**; Legitimate P 0.991 / R 0.992 / F1 0.992 (n=1,999); Phishing P 0.981 / R 0.973 / F1 0.977 (n=521); Spam P 0.977 / R 0.979 / F1 0.978 (n=700). 3x3 (rows true, cols predicted Legitimate/Phishing/Spam): Legitimate 1,984/4/11; Phishing 9/507/5; Spam 9/6/685. Phishing-vs-spam on the 1,221 true unwanted rows: kind accuracy 0.9894 (7 phishing->spam, 6 spam->phishing).
+
+| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |
+|---|---:|---:|---:|---:|---:|---:|
+| Assassin | 236 | 164 | 0 | 72 | 0.9449 | 0.9464 |
+| CEAS-08 | 736 | 736 | 0 | 0 | 0.9905 | 0.9952 |
+| Enron | 628 | 0 | 0 | 628 | 0.9889 | 0.9944 |
+| Nazario | 222 | 0 | 222 | 0 | 0.9505 | 0.9746 |
+| Nigerian_Fraud | 299 | 0 | 299 | 0 | 0.9900 | 0.9950 |
+| TREC-07 | 1,099 | 1,099 | 0 | 0 | 0.9973 | 0.9986 |
+
+**Latency (batch 1, tokenize + forward + softmax, timeit.default_timer) vs the instructor's 74 ms.** GPU bf16: 9.89 ms avg / 8.93 ms p50 over 100 emails (0.13x). CPU fp32 (16 threads): 513.57 ms avg / 485.83 ms p50 over 100 emails (6.94x).
+
+**Length confound (test).** FN mean 256 tokens vs TP 287; FP 763 vs TN 848 (see error_analysis.md for the per-bucket table).
+
+**Artifacts.** `/home/nathank/BITS/dev/Phishing Model/reports`: metrics.json, latency.json, confusion_matrix.png, roc_curve.png, pr_curve.png, error_analysis.md; metrics_test3.json, latency_test3.json, confusion_matrix_test3.png, confusion_matrix3_test3.png, roc_curve_test3.png, pr_curve_test3.png, error_analysis_test3.md (training curves: reports/pretrain_curves.png and finetune_curves.png from pretrain.py / finetune.py `plot`).
+
+## 22. 2026-10-09 10:46 — Evaluation on the test + test3 splits (evaluate.py run, `ft-focal`)
+
+**Checkpoint.** `/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best` (3-class head; three-way label contract of RUN_LOG entry 6). Binary roll-up metrics use the same definitions as the binary-era runs so they stay comparable.
+
+### test split (TREC-06 binary holdout): `test_holdout.parquet`, 16,302 rows (3,894 unwanted, 23.9%; label3 legit 12,408, unwanted 3,894). Inference 284.1 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9741** (101 missed of 3,894) |
+| precision | 0.8794 (520 false alarms) |
+| F1 | 0.9243 |
+| accuracy | 0.9619 |
+| specificity | 0.9581 |
+| ROC-AUC | 0.9946 |
+| PR-AUC | 0.9848 |
+| recall @ 1% FPR | 0.8955 |
+| recall @ 0.1% FPR | 0.6497 |
+| best-F1 threshold | 0.682 (F1 0.9397) |
+| credential-lure slice | 398 rows (61.1% labelled positive), recall 0.9959, precision 0.9758 |
+
+**Three-way:** no phishing-vs-spam ground truth on this split (no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)); informational: of the 3,793 flagged true positives the model called 974 phishing and 2,819 spam.
+
+### test3 split (three-way holdout): `test_holdout3.parquet`, 3,220 rows (1,221 unwanted, 37.9%; label3 legit 1,999, phish 521, spam 700). Inference 237.6 rows/s on cuda (bf16 autocast, batch 64, length-bucketed), peak VRAM 2.27 GB.
+
+| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |
+|---|---:|
+| **recall (unwanted)** | **0.9836** (20 missed of 1,221) |
+| precision | 0.9893 (13 false alarms) |
+| F1 | 0.9864 |
+| accuracy | 0.9898 |
+| specificity | 0.9935 |
+| ROC-AUC | 0.9993 |
+| PR-AUC | 0.9989 |
+| recall @ 1% FPR | 0.9869 |
+| recall @ 0.1% FPR | 0.8788 |
+| best-F1 threshold | 0.475 (F1 0.9877) |
+| credential-lure slice | 275 rows (97.5% labelled positive), recall 1.0000, precision 1.0000 |
+
+**Three-way** on the 3,220 rows with label3 in {0,1,2} (0 unsplit rows enter the roll-up only; rule: rollup_then_argmax): accuracy 0.9885, **macro-F1 0.9867**; Legitimate P 0.990 / R 0.993 / F1 0.992 (n=1,999); Phishing P 0.996 / R 0.987 / F1 0.991 (n=521); Spam P 0.979 / R 0.976 / F1 0.977 (n=700). 3x3 (rows true, cols predicted Legitimate/Phishing/Spam): Legitimate 1,986/1/12; Phishing 4/514/3; Spam 16/1/683. Phishing-vs-spam on the 1,221 true unwanted rows: kind accuracy 0.9959 (4 phishing->spam, 1 spam->phishing).
+
+| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |
+|---|---:|---:|---:|---:|---:|---:|
+| Assassin | 236 | 164 | 0 | 72 | 0.9534 | 0.9458 |
+| CEAS-08 | 736 | 736 | 0 | 0 | 0.9905 | 0.9952 |
+| Enron | 628 | 0 | 0 | 628 | 0.9841 | 0.9920 |
+| Nazario | 222 | 0 | 222 | 0 | 0.9865 | 0.9932 |
+| Nigerian_Fraud | 299 | 0 | 299 | 0 | 0.9866 | 0.9933 |
+| TREC-07 | 1,099 | 1,099 | 0 | 0 | 0.9982 | 0.9991 |
+
+**Latency (batch 1, tokenize + forward + softmax, timeit.default_timer) vs the instructor's 74 ms.** GPU bf16: 10.08 ms avg / 8.9 ms p50 over 100 emails (0.14x). CPU fp32 (16 threads): 458.57 ms avg / 445.01 ms p50 over 100 emails (6.20x).
+
+**Length confound (test).** FN mean 322 tokens vs TP 284; FP 1237 vs TN 827 (see error_analysis.md for the per-bucket table).
+
+**Artifacts.** `/home/nathank/BITS/dev/Phishing Model/reports`: metrics.json, latency.json, confusion_matrix.png, roc_curve.png, pr_curve.png, error_analysis.md; metrics_test3.json, latency_test3.json, confusion_matrix_test3.png, confusion_matrix3_test3.png, roc_curve_test3.png, pr_curve_test3.png, error_analysis_test3.md (training curves: reports/pretrain_curves.png and finetune_curves.png from pretrain.py / finetune.py `plot`).
+
+## 23. 2026-10-12 11:30 — Fine-tune queue results, model selection (`ft-focal`), export
+
+**Queue** (`scripts/run_experiments.sh`, tmux `phish-finetune`): finished 2026-10-09 10:02:22, every step exit 0
+(`ft-ce` 03:36-04:31, `ft-weighted` -05:57, `ft-focal` -07:24, `ft-synthetic` -08:19, `ft-12L` -08:56,
+`ft-scratch` -09:59, then top-level evaluate + export of the pre-designated main run `ft-ce`). Log
+`logs/experiments_20261008-141528.out`. Per-run evaluations in `reports/eval_<run>/` (entries 15-20, 21).
+
+**Selection rule and result.** The queue's main run was `ft-ce` by default, but the rule stated in README section 9
+was "if another run wins on val F1 of the unwanted roll-up, it becomes the main run". `ft-focal` (focal loss,
+gamma 2.0) won on the validation set on both selection metrics: val F1 unwanted 0.9951 vs 0.9929 (`ft-ce`), val
+three-way macro-F1 0.9869 vs 0.9782; it also had the best val phishing precision/recall (0.985 / 0.988). The
+decision was taken from the validation numbers; the holdouts were evaluated once per run for reporting and
+confirm it (TREC-06 F1 0.9243 vs 0.8990, recall at 1% FPR 0.8955 vs 0.7745; test3 macro-F1 0.9867 vs 0.9822).
+Top-level evaluate was re-run for `ft-focal` on 2026-10-09 10:46 (entry 22, `reports/metrics*.json`,
+`latency.json`, plots, `error_analysis*.md`).
+
+**Held-out results of the deployed run `ft-focal`:**
+
+| split | numbers |
+|---|---|
+| TREC-06 (16,302 rows, 23.9% unwanted, whole corpus unseen) | recall 0.9741 (101 missed), precision 0.8794 (520 false alarms, FPR 4.2%), F1 0.9243, accuracy 0.9619, ROC-AUC 0.9946, PR-AUC 0.9848, recall at 1% / 0.1% FPR 0.8955 / 0.6497 |
+| test_holdout3 (3,220 rows: 1,999 / 521 / 700) | macro-F1 0.9867, accuracy 0.9885; F1 legit / phishing / spam 0.9918 / 0.9913 / 0.9771; binary roll-up F1 0.9864, ROC-AUC 0.9993; phishing-vs-spam on true unwanted 0.9959 (4 phishing->spam, 1 spam->phishing); 3x3 [[1986, 1, 12], [4, 514, 3], [16, 1, 683]] |
+| latency, batch 1, 100 TREC-06 emails (mean 280 tokens) | GPU bf16 10.08 ms mean / 8.90 p50 / 13.72 p95 (0.14x the 74 ms benchmark); CPU fp32 16 threads 458.6 ms mean / 445.0 p50 / 701.1 p95 (6.20x) |
+
+**What the ablations say** (all six on the same holdouts, README 9.1): class-weighted CE trades precision for
+recall (TREC-06 recall 0.982 but FPR 6.3%); focal wins on every metric; the 2,000-row synthetic supplement
+changes nothing measurable (TREC-06 F1 0.8995 vs 0.8990; val macro-F1 lower, because its 1,000 phishing lures
+are template-generated); 12 layers lose ~5 F1 points on TREC-06 (0.8739) for ~2x faster inference; scratch
+init loses 8 points (0.8400) and 1.7 ROC-AUC points, which is what the 15 h of MLM pretraining buys.
+
+**Challenge recorded: TREC-06 labelling conventions.** The TREC-06 numbers are the honest, cross-corpus ones and
+sit well below the in-distribution validation numbers (F1 0.92 vs 0.995). `reports/error_analysis.md` shows the
+two causes: (1) TREC-06 labels as *ham* a lot of mail that looks like spam to any text model (spam quoted inside
+mailing-list discussions, "please remove me from your list" replies, sweepstakes mail, "57 Million Email
+Addresses - ONLY $149!!" on a developer list), which accounts for the most confident false positives; (2) its
+largest block of *spam* is "ScanMail Message: virus found" gateway notices, which every run misses. Legitimate
+TREC-06 mail is also ~3x longer than its spam and 24% of it is truncated at 512 tokens; false positives are
+truncated 31.5% of the time vs 23.6% for true negatives. Neither is a leakage problem, but both cap what any
+text-only classifier can score on this corpus, and they are the right thing to show in the presentation when
+explaining the gap between 0.995 validation and 0.92 TREC-06.
+
+**Export.** 2026-10-12 11:28: `export.py run --checkpoint checkpoints/finetune/ft-focal/best --out-dir
+model_export/ft-focal --app-dir app` -> `model_export/ft-focal/` (bf16 safetensors 674.9 MB, config.json with
+labels + metrics + dials, tokenizer.json, model-card README.md, export_manifest.json) and the same three files
+into `app/`; `app/model.py` is byte-identical to `src/model.py` (sha1 39b3773c, asserted). This replaced the
+`ft-ce` weights that the queue had copied into `app/` on 10-09 10:02; `model_export/ft-ce/` is kept for
+comparison. README (sections 2.1, 2.2, 5.6-5.8, 9, 10, 11), `app/README.md` (Space card: bundled weights,
+training recipe, real model-card link) and `scripts/build_notebook.py` were updated to name `ft-focal` as the
+deployed run; the notebook was rebuilt.
+
+**Gap in the record.** The 2026-10-09 10:43 scheduled check-in completed the queue review and the `ft-focal`
+top-level evaluation (entry 22) but stalled before export/upload; resumed on 2026-10-12 11:27 at Nathan's
+request. `find -newermt` confirmed nothing in the project changed in between.
+
+## 24. 2026-10-12 11:35 — Hugging Face upload (private model repo + private ZeroGPU Space)
+
+Approved by Nathan on 2026-10-08 19:15 ("can you upload the hugging face?"); deferred until an export existed.
+Account: `BuildITSystems` via the `hf auth login` cache (token never printed or written anywhere).
+
+**Dry run** 11:30 (`scripts/upload_to_hf.py --model-dir model_export/ft-focal --bundle-weights`): no problems;
+plan = 4 files / 645.9 MiB to the model repo, 8 files / 645.9 MiB to the Space (weights bundled, so no
+`HF_TOKEN` secret), hardware `zero-a10g`, both private.
+
+**Live run 1** 11:31 (`--yes`): `create_repo` + `upload_folder` for the model repo succeeded (commit
+`80bd85c5`: README.md model card, config.json, model.safetensors 674.9 MB bf16, tokenizer.json). The Space was
+created and its `PHISH_MODEL_REPO` variable set, but `upload_folder(app/)` was refused by the Hub's README
+validation: `Invalid YAML in README.md: bad indentation of a mapping entry (11:45)` — the unquoted
+`short_description: From-scratch 337M encoder: legitimate / phishing / spam` contains a colon. The dry run does
+not validate the card's YAML, so this was invisible until the live call. Exit 1.
+
+**Fix** 11:34: quoted the value in `app/README.md` (55 chars, under the 60-char limit); verified with
+`yaml.safe_load` and `huggingface_hub.RepoCard`. Lesson added to README 5.9.
+
+**Live run 2** 11:35 (`--yes` again, idempotent): model-repo files already present (LFS content dedupe; commit
+unchanged), Space `upload_folder` succeeded (commit `98f90d7b`). Verified read-only afterwards:
+
+| repo | private | files |
+|---|---|---|
+| https://huggingface.co/BuildITSystems/phishing-detector-encoder | yes | .gitattributes, README.md, config.json, model.safetensors 674.9 MB, tokenizer.json 2.3 MB |
+| https://huggingface.co/spaces/BuildITSystems/phishing-detector-demo | yes | .gitattributes, README.md (card), app.py, model.py, requirements.txt, config.json, model.safetensors 674.9 MB, tokenizer.json 2.3 MB |
+
+Space runtime at 11:36: stage `BUILDING`, requested hardware `zero-a10g`; variable
+`PHISH_MODEL_REPO=BuildITSystems/phishing-detector-encoder`; no secrets. Deployed weights = `ft-focal`
+(entry 23). Before class Nathan flips both repos to public (README section 10 one-liner) and opens the Space
+once so ZeroGPU is warm.
+
+**Build and live check (2026-10-12 11:37-11:39).** Space stages: BUILDING 11:35 -> APP_STARTING 11:36 -> RUNNING
+on `zero-a10g` 11:37 (about 2 minutes end to end). Verified through the private Space's Gradio API
+(`gradio_client`, endpoint `/predict(subject, body)`, token from the login cache): a sample credential-lure
+email ("verify your account within 24 hours", lookalike PayPal link) returned verdict **Phishing** 95.6%,
+Spam 3.5%, Legitimate 0.9%, roll-up Unwanted 99.1%, in-app "Prediction Time (s)" 0.253 (cold ZeroGPU first
+call), round-trip 3.6 s, with the token highlights and the rule-based signals populated. The deployment is
+working end to end.
diff --git a/presentation/app/.gitattributes b/presentation/app/.gitattributes
new file mode 100644
index 0000000000000000000000000000000000000000..a698f3f62ec5c0873fb278bccf7fc224cf5dfc21
--- /dev/null
+++ b/presentation/app/.gitattributes
@@ -0,0 +1,4 @@
+# Large binaries go through LFS/Xet. The Hub assigns LFS automatically on HTTP upload (upload_folder);
+# this file records the intent for anyone who clones the Space or model repo with git.
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
diff --git a/presentation/app/README.md b/presentation/app/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..5f2b36017af3e446234847ba39e62f8f851aa419
--- /dev/null
+++ b/presentation/app/README.md
@@ -0,0 +1,79 @@
+---
+title: BITS Phishing Email Detector
+emoji: 🛡️
+colorFrom: blue
+colorTo: indigo
+sdk: gradio
+sdk_version: 6.29.1
+python_version: "3.12.12"
+app_file: app.py
+pinned: false
+license: mit
+short_description: "From-scratch 337M encoder: legitimate / phishing / spam"
+---
+
+# BITS Phishing Email Detector
+
+Paste an email subject and body; the app returns the model's probability for each of three classes,
+**Legitimate**, **Phishing** and **Spam**, plus the roll-up **Unwanted (phishing or spam)** = P(Phishing) +
+P(Spam), the prediction time for that single sample, the tokens the model weighed most, and a list of
+rule-based cues for a human reader. The verdict is Legitimate when the unwanted roll-up is below 50% and
+otherwise the larger of Phishing / Spam. It is a defensive triage and training aid.
+
+## How it works
+
+- **Model.** A 337M-parameter BERT-style pre-LayerNorm encoder (24 layers, d=1024, 16 heads, FFN 4096,
+  vocabulary 32,768, 512 tokens) written from scratch in plain PyTorch (`model.py`; no `transformers`, no
+  pretrained weights) with a three-way classification head. It was pretrained with masked language modelling
+  on deduplicated Enron email and fine-tuned on public corpora: legitimate mail and spam from TREC-05/07,
+  CEAS-08, Enron-Spam and SpamAssassin; phishing from the Nazario phishing archive and the Nigerian-fraud
+  (advance-fee) archive (Champa et al. corpus mirror, CC-BY-4.0) plus BITS synthetic lures. Rows whose corpus
+  never separated spam from phishing (TREC-05/07, CEAS-08 positives) were trained with a partial label on the
+  combined phishing + spam mass, so they shape the legitimate-vs-unwanted decision without asserting which
+  unwanted class they belong to. The TREC-06 corpus was held out entirely as the binary test set; a separate
+  three-way holdout of rows the tokenizer never saw and that share no exact or near-duplicate text with the
+  pretraining corpus (checked by content keys and shared 50-character shingles) gives the 3x3 confusion matrix.
+- **Input.** `subject + "\n\n" + body`, byte-level BPE tokenised (NFKC, case preserved) and truncated to 512
+  tokens. Sender, headers and attachments are never read.
+- **Prediction time.** `timeit.default_timer` around tokenize + forward + softmax, like the course reference
+  app (benchmark: 0.074 s/sample).
+- **Token importance.** Gradient x input saliency of the verdict's log-probability (log P(Legitimate), or
+  log(P(Phishing) + P(Spam)) when the verdict is unwanted) with respect to the token embeddings; the top tokens
+  are merged back into words and highlighted over the original text.
+- **Signals.** Regex heuristics (urgency words, credential lures, link-text vs href host mismatch, lookalike
+  domains, raw-IP URLs, ALL-CAPS share, reply-to redirection, generic greetings, risky attachment names).
+  They are shown for context only; the probability comes from the model alone.
+- **Runtime.** ZeroGPU (`@spaces.GPU(duration=30)`, bf16) with the model placed on `cuda` at module level;
+  falls back to CPU fp32. The weights (`model.safetensors`, `config.json`, `tokenizer.json`) are bundled in
+  this Space next to `app.py`; `app.py` can also download them from the model repo named by the
+  `PHISH_MODEL_REPO` variable with an `HF_TOKEN` secret.
+- **Training recipe of the deployed checkpoint.** Fine-tune run `ft-focal`: focal loss (gamma 2.0) over the
+  partial-label negative log-likelihood, AdamW, LR 3e-5 with warmup and linear decay, batch 32, early stopping
+  on the validation F1 of the unwanted roll-up. It was chosen among six runs (cross-entropy, class-weighted
+  cross-entropy, focal, synthetic supplement, 12 layers, scratch init) by validation metrics alone.
+
+## Model card
+
+Weights, config, tokenizer, metrics and the full model card: https://huggingface.co/BuildITSystems/phishing-detector-encoder
+
+## Limitations
+
+- Training data is English email from 1995-2022, the bulk of it 1998-2008 (Enron, TREC, CEAS, SpamAssassin);
+  modern lures (MFA fatigue, QR codes, chat-app phishing) and non-English mail are out of distribution.
+- **Spam vs phishing follows corpus provenance, not a per-message review.** "Phishing" means a row came from
+  the Nazario or Nigerian-fraud archives or the synthetic lures; "Spam" means the SpamAssassin or Enron-Spam
+  spam folders. The TREC-05/07 and CEAS-08 positives (the largest share of unwanted training mail) were never
+  split and were trained only as "unwanted", so the phishing/spam boundary is softer than the
+  legitimate/unwanted one. The roll-up **Unwanted (phishing or spam)** is the number to trust first and the one
+  comparable with the earlier binary runs.
+- Text only: no sender reputation, SPF/DKIM, link reputation or attachment analysis.
+- Inputs longer than 512 tokens (about 1,800 characters) are judged on their opening.
+- Legitimate mail is on average longer than spam in the training data; length is a known confound.
+- Flagging is disabled and nothing pasted is stored, but do not paste secrets or confidential client content.
+
+## Context
+
+Final project for the Skillspire AI & Machine Learning course, built by Nathan King at Build IT Systems (BITS).
+The project repository holds the data pipeline, tokenizer, pretraining and fine-tuning code, the run log of
+every experiment (layers, epochs, loss functions, crashes and fixes) and the evaluation on the unseen TREC-06
+test set; the headline numbers are in the model card linked above.
diff --git a/presentation/app/app.py b/presentation/app/app.py
new file mode 100644
index 0000000000000000000000000000000000000000..832e000fa4e68a22382adbc1bb3503972acbc7b1
--- /dev/null
+++ b/presentation/app/app.py
@@ -0,0 +1,901 @@
+"""BITS Phishing Email Detector: Gradio demo for the from-scratch 337M encoder classifier.
+
+Mirrors the instructor's reference app (``gr.Interface``, ``gr.Label`` + ``gr.Number``
+"Prediction Time (s)`` timed with ``timeit.default_timer``) for text instead of images,
+and adds two interpretability outputs:
+
+* **Three classes.** The head is a softmax over ``Legitimate / Phishing / Spam`` (names
+  read from ``config.json["labels"]``; a two-class export falls back to the old
+  ``Legitimate / Spam/Phishing`` pair). The binary roll-up ``P(unwanted) = P(phishing) +
+  P(spam)`` is the headline: it is the quantity the TREC-era training corpora actually
+  label, so it stays comparable with the binary runs. The verdict is ``Legitimate`` when
+  ``P(unwanted) < 0.5``, otherwise whichever unwanted class carries more mass.
+* **Model-based token importance** (gradient x input saliency). The log-probability of
+  the verdict (``log P(legitimate)``, or ``log(P(phishing) + P(spam))`` when the verdict is
+  unwanted) is differentiated with respect to the token embedding vectors; the dot
+  product of gradient and embedding per position is a first-order estimate of how much
+  that token moved the decision toward the verdict. A log-probability rather than a raw
+  logit is used because softmax is shift-invariant: a bias shared by all logits would
+  otherwise show up as spurious importance. Saliency is faithful to this model (it reads
+  the model's own gradients) where attention weights are not reliably so.
+* **Rule-based cues**: regex heuristics (urgency words, credential lures, link-text /
+  href host mismatch, lookalike domains, IP-literal URLs, ALL-CAPS share, reply-to
+  redirection, generic greetings, risky attachment names). These are shown as context
+  for a human reader and never feed the probability, which comes only from the model.
+
+Runtime layout (Hugging Face Space, ZeroGPU): ``import spaces`` runs before torch so the
+CUDA shim is installed first; the model is moved to ``cuda`` at MODULE level (ZeroGPU
+patches ``torch.cuda.is_available()`` to True outside ``@spaces.GPU`` functions and
+requires module-level placement); the only CUDA-touching function is wrapped in
+``@spaces.GPU(duration=30)``. Locally (no ``spaces`` package or ``SPACES_ZERO_GPU``
+unset) the decorator is a no-op and the app runs on the local GPU or on the CPU.
+
+Weights: ``PHISH_MODEL_DIR`` (default: this file's directory) is used when it holds
+``model.safetensors`` + ``config.json`` + ``tokenizer.json``; otherwise the three files
+are downloaded from the private model repo ``PHISH_MODEL_REPO`` with the ``HF_TOKEN``
+Space secret (``preload_from_hub`` cannot read private repos). The token is read from
+the environment only and never logged.
+
+Hidden flags (development): ``--selftest`` runs ``predict`` on three built-in emails (a
+phishing lure, a marketing spam, a legitimate note) on CPU and on CUDA (if present) and
+exits; ``--serve-seconds N`` launches the server without blocking, fetches the root URL
+once, waits N seconds and exits.
+"""
+
+from __future__ import annotations
+
+import argparse
+import html
+import json
+import logging
+import math
+import os
+import re
+import sys
+import time
+from pathlib import Path
+from timeit import default_timer as timer
+from typing import Any
+
+TITLE = "BITS Phishing Email Detector"
+
+
+def _parse_args(argv: list[str] | None = None) -> argparse.Namespace:
+    ap = argparse.ArgumentParser(description=TITLE, add_help=True)
+    ap.add_argument("--selftest", action="store_true", help=argparse.SUPPRESS)
+    ap.add_argument("--serve-seconds", type=float, default=None, help=argparse.SUPPRESS)
+    return ap.parse_args(argv)
+
+
+# Parsed before the heavy imports so ``--help`` (or a bad flag) answers without importing
+# torch, model.py or the weights. Spaces runs ``python app.py`` with no arguments.
+ARGS = _parse_args() if __name__ == "__main__" else None
+
+try:  # must precede torch: the ZeroGPU shim patches torch.cuda at import time
+    import spaces
+except ImportError:  # local run without the ``spaces`` package
+
+    class _SpacesShim:
+        """``@spaces.GPU`` and ``@spaces.GPU(duration=..)`` both become identity decorators."""
+
+        @staticmethod
+        def GPU(task=None, *, duration=None, size=None):  # noqa: N802 - mirrors spaces.GPU
+            if callable(task):
+                return task
+            return lambda f: f
+
+    spaces = _SpacesShim()  # type: ignore[assignment]
+
+import torch  # noqa: E402
+import torch.nn.functional as F  # noqa: E402
+
+APP_DIR = Path(__file__).resolve().parent
+MODEL_DIR = Path(os.environ.get("PHISH_MODEL_DIR", str(APP_DIR))).resolve()
+MODEL_REPO = os.environ.get("PHISH_MODEL_REPO", "BuildITSystems/phishing-detector-encoder")
+WEIGHT_FILES = ("model.safetensors", "config.json", "tokenizer.json")
+ZERO_GPU = os.environ.get("SPACES_ZERO_GPU", "").lower() in {"1", "t", "true"}
+
+# model.py normally sits next to app.py (export.py copies it there). A development export
+# directory may carry its own copy; it is searched second so the Space's copy wins.
+for _p in (APP_DIR, MODEL_DIR):
+    if str(_p) not in sys.path:
+        sys.path.append(str(_p))
+
+import gradio as gr  # noqa: E402
+from safetensors.torch import load_file  # noqa: E402
+from tokenizers import Tokenizer  # noqa: E402
+
+from model import EncoderConfig, EncoderForClassification  # noqa: E402
+
+logging.basicConfig(
+    level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s", datefmt="%H:%M:%S", stream=sys.stdout
+)
+LOG = logging.getLogger("phish_app")
+
+# Display names by class index. Three classes since Nathan's 2026-10-08 scope change (RUN_LOG
+# entry 6); the two-class names are kept so an older binary export still renders correctly.
+DEFAULT_LABELS: dict[int, list[str]] = {
+    2: ["Legitimate", "Spam/Phishing"],
+    3: ["Legitimate", "Phishing", "Spam"],
+}
+UNWANTED_LABEL = "Unwanted (phishing or spam)"   # the binary roll-up P(class 1) + P(class 2) + ...
+LEGIT_INDEX = 0                                  # every class >= 1 is "unwanted" (contract: 1 phishing, 2 spam)
+TOP_K_TOKENS = 12
+HIGH_FRACTION = 0.5  # a selected word is "high" when |score| >= HIGH_FRACTION * max |score|
+
+
+def label_names(config: dict[str, Any], num_labels: int) -> list[str]:
+    """Class display names from ``config.json["labels"]`` (``{"0": "Legitimate", ...}``).
+
+    The export writes the names the fine-tune was trained with; they win when they cover
+    every index with a distinct non-empty string. Otherwise the defaults above apply
+    (``num_labels == 2`` -> the binary pair, 3 -> Legitimate / Phishing / Spam).
+    """
+    raw = config.get("labels")
+    if isinstance(raw, (list, tuple)):  # tolerate a positional list ["Legitimate", "Phishing", "Spam"]
+        raw = {str(i): v for i, v in enumerate(raw)}
+    if not isinstance(raw, dict):  # missing, null, a bare string, ... -> defaults, never an import-time crash
+        raw = {}
+    names = [str(raw.get(str(i), raw.get(i, "")) or "").strip() for i in range(num_labels)]
+    if all(names) and len(set(names)) == num_labels:
+        return names
+    return DEFAULT_LABELS.get(num_labels) or [f"Class {i}" for i in range(num_labels)]
+
+
+# ----------------------------------------------------------------------------- weights
+def resolve_model_files() -> tuple[dict[str, Path], str]:
+    """Local files in MODEL_DIR win; otherwise download from the private model repo.
+
+    Returns ``({filename: path}, source)``. The HF token is passed straight from the
+    environment to ``hf_hub_download`` and never echoed.
+    """
+    local = {f: MODEL_DIR / f for f in WEIGHT_FILES}
+    if all(p.is_file() for p in local.values()):
+        return local, f"local:{MODEL_DIR}"
+    missing = [f for f, p in local.items() if not p.is_file()]
+    LOG.info("weights not found locally (missing %s in %s); downloading from %s", missing, MODEL_DIR, MODEL_REPO)
+    from huggingface_hub import hf_hub_download
+
+    token = os.environ.get("HF_TOKEN")
+    LOG.info("HF_TOKEN %s", "present" if token else "absent (public repo access only)")
+    files = {f: Path(hf_hub_download(repo_id=MODEL_REPO, filename=f, token=token)) for f in WEIGHT_FILES}
+    return files, f"hub:{MODEL_REPO}"
+
+
+def load_model(files: dict[str, Path]) -> tuple[EncoderForClassification, EncoderConfig, dict[str, Any]]:
+    """Build ``EncoderForClassification`` from config.json and load the safetensors strictly.
+
+    The export is bf16; ``load_state_dict`` copies into the fp32 parameters of the freshly
+    built module, so the dtype actually used at inference is decided afterwards in
+    ``place_model`` (bf16 on CUDA, fp32 on CPU).
+    """
+    with open(files["config.json"], encoding="utf-8") as fh:
+        config = json.load(fh)
+    cfg = EncoderConfig.from_dict(config["model"])
+    model = EncoderForClassification(cfg)
+    state = load_file(str(files["model.safetensors"]), device="cpu")
+    model.load_state_dict(state, strict=True)
+    model.eval()
+    for p in model.parameters():  # inference only; saliency differentiates w.r.t. embeddings, not weights
+        p.requires_grad_(False)
+    return model, cfg, config
+
+
+def place_model(model: torch.nn.Module, device: str) -> torch.nn.Module:
+    """bf16 on CUDA (halves memory and bandwidth; the export is bf16 anyway), fp32 on CPU
+    (CPU bf16 matmuls are slow on most hosts and fp32 is the reference numerics)."""
+    if device == "cuda":
+        return model.to(device="cuda", dtype=torch.bfloat16)
+    return model.float().to("cpu")
+
+
+_t0 = timer()
+_FILES, WEIGHT_SOURCE = resolve_model_files()
+MODEL, CFG, CONFIG = load_model(_FILES)
+MAX_LEN = int(CFG.max_seq_len)
+# Two tokenizer instances with FIXED settings instead of one whose truncation is toggled per
+# call: the Rust tokenizer's truncation state is shared mutable state, so toggling it would
+# race if Gradio's concurrency_limit were ever raised above 1. TOKENIZER always truncates to
+# MAX_LEN (the model input); TOKENIZER_FULL never does (the untruncated count for the notice).
+TOKENIZER = Tokenizer.from_file(str(_FILES["tokenizer.json"]))
+TOKENIZER.no_padding()
+TOKENIZER.enable_truncation(max_length=MAX_LEN)
+TOKENIZER_FULL = Tokenizer.from_file(str(_FILES["tokenizer.json"]))
+TOKENIZER_FULL.no_padding()
+TOKENIZER_FULL.no_truncation()
+NUM_LABELS = int(CFG.num_labels)
+if NUM_LABELS < 2:
+    raise RuntimeError(f"config.json model.num_labels={NUM_LABELS}; the app needs at least a legitimate/unwanted pair")
+LABELS: list[str] = label_names(CONFIG, NUM_LABELS)
+DEVICE = "cuda" if torch.cuda.is_available() else "cpu"  # patched True under ZeroGPU, by design
+MODEL = place_model(MODEL, DEVICE)
+N_PARAMS = sum(p.numel() for p in MODEL.parameters())
+LOG.info(
+    "model loaded in %.1fs from %s: %s params, %d layers, %d classes %s, device=%s dtype=%s, zero_gpu=%s, max_len=%d",
+    timer() - _t0, WEIGHT_SOURCE, f"{N_PARAMS:,}", CFG.n_layers, NUM_LABELS, LABELS, DEVICE,
+    next(MODEL.parameters()).dtype, ZERO_GPU, MAX_LEN,
+)
+
+
+# ----------------------------------------------------------------------------- inference
+def _autocast(device: str):
+    if device == "cuda":
+        return torch.autocast("cuda", dtype=torch.bfloat16)
+    return torch.autocast("cpu", enabled=False)
+
+
+def encode(text: str) -> dict[str, Any]:
+    """[CLS] text [SEP], truncated to MAX_LEN; offsets refer to the ORIGINAL string
+    (the tokenizers library keeps alignments through NFKC), which is what lets the
+    saliency be painted over the user's text rather than over a normalised copy.
+    Stateless: TOKENIZER is configured once at import, nothing is toggled here."""
+    enc = TOKENIZER.encode(text)
+    return {
+        "ids": enc.ids,
+        "offsets": enc.offsets,
+        "special": enc.special_tokens_mask,
+        "n_tokens": len(enc.ids),
+        "truncated": len(enc.overflowing) > 0,  # truncation leaves the cut-off part in ``overflowing``
+    }
+
+
+def full_token_count(text: str) -> int:
+    """Untruncated token count for the truncation notice. Called only after the prediction
+    timer has stopped and only when ``encode`` reported truncation, so a long paste never
+    inflates the graded Prediction Time with a second tokenizer pass."""
+    return len(TOKENIZER_FULL.encode(text).ids)
+
+
+def verdict_from_probs(probs: list[float]) -> tuple[int, float]:
+    """``(verdict_index, p_unwanted)``: Legitimate when the unwanted mass is below 0.5,
+    otherwise the unwanted class with the most mass (phishing vs spam is decided only
+    among the unwanted classes, so a 0.40/0.35/0.25 split still reads as unwanted)."""
+    p_unwanted = float(sum(probs[LEGIT_INDEX + 1:]))
+    if p_unwanted < 0.5:
+        return LEGIT_INDEX, p_unwanted
+    top = max(range(LEGIT_INDEX + 1, len(probs)), key=lambda i: probs[i])
+    return top, p_unwanted
+
+
+@spaces.GPU(duration=30)
+def _infer(text: str) -> dict[str, Any]:
+    """Tokenize + forward + softmax (timed like the instructor's app), then one extra
+    forward/backward for gradient x input saliency. Runs on ``DEVICE``; returns only
+    plain Python data so it crosses the ZeroGPU process boundary cleanly."""
+    device = DEVICE
+    start = timer()
+    enc = encode(text)
+    ids = torch.tensor([enc["ids"]], dtype=torch.int64, device=device)
+    mask = torch.ones_like(ids, dtype=torch.bool)
+    with torch.inference_mode(), _autocast(device):
+        logits = MODEL(ids, mask).float()
+        probs = [float(v) for v in torch.softmax(logits, dim=-1)[0].tolist()]  # .tolist() syncs CUDA before the timer stops
+    elapsed = timer() - start
+    if not all(math.isfinite(p) for p in probs):
+        # NaN/inf logits (corrupted weights, bf16 overflow) must surface as an error: ``nan < 0.5``
+        # is False, so verdict_from_probs would otherwise announce a confident "Phishing" at nan%.
+        raise gr.Error("The model produced non-finite probabilities for this input; the weights or the input are corrupt.")
+    verdict, p_unwanted = verdict_from_probs(probs)
+
+    full_len = full_token_count(text) if enc["truncated"] else enc["n_tokens"]  # outside the timed window
+    t1 = timer()
+    saliency = _gradient_x_input(ids, mask, device, unwanted=verdict != LEGIT_INDEX)
+    sal_elapsed = timer() - t1
+    return {
+        "probs": probs,
+        "p_unwanted": p_unwanted,
+        "verdict": verdict,
+        "elapsed_s": elapsed,
+        "saliency_s": sal_elapsed,
+        "saliency": saliency,
+        "offsets": enc["offsets"],
+        "special": enc["special"],
+        "n_tokens": enc["n_tokens"],
+        "truncated": enc["truncated"],
+        "full_len": full_len,
+        "device": device,
+        "dtype": str(next(MODEL.parameters()).dtype).replace("torch.", ""),
+    }
+
+
+def _verdict_log_prob(logits: torch.Tensor, unwanted: bool) -> torch.Tensor:
+    """Scalar whose gradient the saliency reads: ``log P(unwanted)`` = ``logsumexp(logits[1:])
+    - logsumexp(logits)`` when the verdict is unwanted (the mass of phishing + spam, so
+    the two unwanted classes are not played against each other), else ``log P(legit)``.
+    Both are shift-invariant in the logits, unlike a raw class logit."""
+    row = logits[0]
+    lse_all = torch.logsumexp(row, dim=-1)
+    if unwanted:
+        return torch.logsumexp(row[LEGIT_INDEX + 1:], dim=-1) - lse_all
+    return row[LEGIT_INDEX] - lse_all
+
+
+def _gradient_x_input(ids: torch.Tensor, mask: torch.Tensor, device: str, *, unwanted: bool) -> list[float]:
+    """Per-token ``grad(log P(verdict)) . embedding`` with gradients enabled only for this call.
+
+    A forward hook swaps the token-embedding output for a detached leaf that requires
+    grad, so autograd builds a graph from that tensor onward only: no parameter
+    gradients are accumulated and the backward is as cheap as the model allows.
+    Positive values push toward the verdict, negative away from it.
+    """
+    captured: dict[str, torch.Tensor] = {}
+
+    def hook(_module, _inputs, output):
+        leaf = output.detach().requires_grad_(True)
+        captured["emb"] = leaf
+        return leaf
+
+    handle = MODEL.encoder.embeddings.tok.register_forward_hook(hook)
+    try:
+        with torch.enable_grad(), _autocast(device):
+            logits = MODEL(ids, mask).float()
+            target = _verdict_log_prob(logits, unwanted)
+            (grad,) = torch.autograd.grad(target, captured["emb"])
+    finally:
+        handle.remove()
+    sal = (grad.float() * captured["emb"].float()).sum(-1)[0]
+    return [float(v) for v in sal.tolist()]
+
+
+# ----------------------------------------------------------------------------- saliency -> words
+WORD_BREAK = set(" \t\r\n\f\v<>\"'=")  # whitespace plus HTML attribute syntax; URLs stay whole
+
+
+def _is_break(ch: str) -> bool:
+    return ch.isspace() or ch in WORD_BREAK
+
+
+def _word_span(text: str, start: int, end: int) -> tuple[int, int] | None:
+    """Trim break characters inside a token span, then widen to the enclosing word.
+
+    Words are delimited by whitespace and by ``< > " ' =`` so that ``href="http://x">text</a>``
+    splits into its parts while a URL or an e-mail address is kept as one unit."""
+    while start < end and _is_break(text[start]):
+        start += 1
+    while end > start and _is_break(text[end - 1]):
+        end -= 1
+    if start >= end:
+        return None
+    while start > 0 and not _is_break(text[start - 1]):
+        start -= 1
+    while end < len(text) and not _is_break(text[end]):
+        end += 1
+    return start, end
+
+
+def saliency_highlights(text: str, out: dict[str, Any], k: int = TOP_K_TOKENS) -> list[tuple[str, str | None]]:
+    """Top-k tokens by saliency in the direction of the verdict, merged into words and
+    rendered as ``gr.HighlightedText`` segments over the original text.
+
+    The saliency target is already the verdict's log-probability, so positive scores
+    answer "what pushed the model to this answer"; tokens pulling the other way are left
+    unpainted. Scores of tokens merged into one word are summed, so a word split into
+    several sub-tokens is weighted as a whole.
+    """
+    scored: list[tuple[float, tuple[int, int]]] = []
+    for s, (a, b), special in zip(out["saliency"], out["offsets"], out["special"]):
+        if special or b <= a:
+            continue
+        span = _word_span(text, a, b)
+        if span is None:
+            continue
+        scored.append((s, span))
+    scored = [t for t in scored if t[0] > 0]
+    scored.sort(key=lambda t: -t[0])
+    chosen = scored[:k]
+    if not chosen:
+        return [(text, None)]
+    # merge overlapping / identical word spans (sub-tokens of one word, repeated words)
+    merged: dict[tuple[int, int], float] = {}
+    for score, span in chosen:
+        merged[span] = merged.get(span, 0.0) + score
+    spans = sorted(merged.items())
+    flat: list[tuple[int, int, float]] = []
+    for (a, b), sc in spans:
+        if flat and a <= flat[-1][1]:
+            pa, pb, ps = flat[-1]
+            flat[-1] = (pa, max(pb, b), ps + sc)
+        else:
+            flat.append((a, b, sc))
+    top = max(sc for _, _, sc in flat)
+    segments: list[tuple[str, str | None]] = []
+    cursor = 0
+    for a, b, sc in flat:
+        if a > cursor:
+            segments.append((text[cursor:a], None))
+        segments.append((text[a:b], "high" if sc >= HIGH_FRACTION * top else "medium"))
+        cursor = b
+    if cursor < len(text):
+        segments.append((text[cursor:], None))
+    return segments
+
+
+# ----------------------------------------------------------------------------- rule-based cues
+# Shared with src/evaluate.py (credential-lure slice of the holdout): keep the two copies
+# identical; the app directory is self-contained and cannot import from src/.
+CREDENTIAL_LURE_PHRASES: tuple[str, ...] = (
+    "verify your account", "verify your identity", "confirm your account", "confirm your identity",
+    "confirm your password", "update your password", "reset your password", "password will expire",
+    "password expires", "password has expired", "update your payment", "update your billing",
+    "billing information", "unusual sign-in", "unusual activity", "suspicious activity",
+    "account has been suspended", "account will be suspended", "account has been limited",
+    "unlock your account", "re-activate your account", "reactivate your account", "validate your account",
+    "login to your account", "log in to your account", "sign in to your account", "click here to verify",
+    "security alert", "security notice", "enter your password", "re-enter your", "social security number",
+    "credit card number", "one-time passcode", "verification code",
+)
+URGENCY_WORDS: tuple[str, ...] = (
+    "urgent", "immediately", "immediate action", "within 24 hours", "within 48 hours", "final notice",
+    "final warning", "last warning", "act now", "right away", "as soon as possible", "asap", "expires today",
+    "will expire", "will be closed", "will be terminated", "will be deleted", "limited time", "time sensitive",
+    "do not ignore", "failure to", "before it is too late", "today only", "respond now",
+)
+GENERIC_GREETINGS: tuple[str, ...] = (
+    "dear customer", "dear valued customer", "dear user", "dear account holder", "dear member",
+    "dear client", "dear sir/madam", "dear sir or madam", "dear friend", "dear beneficiary",
+    "dear email user", "dear account user", "dear employee", "attention:", "hello dear", "greetings of the day",
+)
+BRANDS: tuple[str, ...] = (
+    "microsoft", "office365", "office 365", "outlook", "onedrive", "sharepoint", "paypal", "apple", "icloud",
+    "amazon", "google", "gmail", "netflix", "chase", "wellsfargo", "bankofamerica", "citibank", "hsbc",
+    "barclays", "dhl", "fedex", "ups", "usps", "docusign", "dropbox", "adobe", "linkedin", "facebook",
+    "instagram", "whatsapp", "irs", "hmrc", "zoom", "webex", "quickbooks", "xero",
+)
+RISKY_ATTACHMENT_EXT = r"zip|rar|7z|htm|html|iso|img|docm|xlsm|pptm|js|vbs|scr|exe|lnk|hta|cmd|bat|msi"
+LEET = str.maketrans({"0": "o", "1": "l", "3": "e", "4": "a", "5": "s", "7": "t", "@": "a", "$": "s"})
+KNOWN_BRAND_DOMAINS = frozenset({
+    "microsoftonline.com", "microsoft.com", "office.com", "live.com", "outlook.com", "windows.net", "sharepoint.com",
+    "onedrive.com", "paypal.com", "apple.com", "icloud.com", "amazon.com", "amazonaws.com", "google.com", "gmail.com",
+    "googleapis.com", "netflix.com", "chase.com", "wellsfargo.com", "bankofamerica.com", "dhl.com", "fedex.com",
+    "ups.com", "usps.com", "docusign.com", "docusign.net", "dropbox.com", "adobe.com", "linkedin.com", "facebook.com",
+    "zoom.us", "webex.com", "irs.gov", "hmrc.gov.uk", "googleusercontent.com", "gstatic.com",
+    "dropboxusercontent.com", "paypalobjects.com", "whatsapp.net", "sharepointonline.com", "intuit.com",
+})
+SHORTENERS = ("bit.ly", "tinyurl.com", "t.co", "goo.gl", "ow.ly", "is.gd", "buff.ly", "cutt.ly", "rb.gy")
+# Extensions that are also everyday technical nouns (node.js, index.html, disk.img, build.cmd):
+# reported only when attachment wording appears within ATTACHMENT_WINDOW characters.
+AMBIGUOUS_ATTACHMENT_EXT = frozenset({"js", "htm", "html", "img", "cmd"})
+ATTACHMENT_WINDOW = 60
+ATTACHMENT_CONTEXT_RE = re.compile(
+    r"(?i)\b(?:attach(?:ed|ment|ments)?|files?|download(?:ed)?|open|enclosed|document|invoice|receipt)\b"
+)
+SYNTHETIC_MARK_RE = re.compile(r"(?i)^\s*\[synthetic\]\s*")
+
+URL_RE = re.compile(r"(?i)\b(?:https?://|www\.)[^\s<>\"')\]]+")
+ANCHOR_RE = re.compile(r"(?is)<a\b[^>]*?href\s*=\s*[\"']?([^\"'\s>]+)[^>]*>(.*?)</a>")
+IP_URL_RE = re.compile(r"(?i)\bhttps?://(?:\d{1,3}\.){3}\d{1,3}(?::\d+)?(?:/|\b)")
+EMAIL_RE = re.compile(r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}")
+REPLY_REDIRECT_RE = re.compile(
+    r"(?i)\b(?:reply|respond|write|send|forward|contact|email|e-mail|get back)\b[^.\n]{0,60}?"
+    r"([A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,})"
+)
+REPLY_TO_HEADER_RE = re.compile(r"(?im)^\s*reply-to\s*:\s*(\S+@\S+)")
+ATTACHMENT_RE = re.compile(rf"(?i)\b[\w\-.]{{1,60}}\.({RISKY_ATTACHMENT_EXT})\b")
+TAG_RE = re.compile(r"<[^>]+>")
+CC_SLD = {"co", "com", "org", "net", "ac", "gov", "edu"}
+
+
+def _host(url: str) -> str:
+    u = url.strip()
+    u = re.sub(r"(?i)^https?://", "", u)
+    host = u.split("/", 1)[0].split("?", 1)[0].split("#", 1)[0]
+    host = host.rsplit("@", 1)[-1].split(":", 1)[0]
+    return host.strip(".,;:!?").lower()  # URL_RE may swallow trailing sentence punctuation
+
+
+def _registrable(host: str) -> str:
+    """Approximate eTLD+1 without a public-suffix list: 'a.b.example.co.uk' -> 'example.co.uk'."""
+    labels = host.split(".")
+    if len(labels) >= 3 and labels[-2] in CC_SLD and len(labels[-1]) == 2:
+        return ".".join(labels[-3:])
+    return ".".join(labels[-2:]) if len(labels) >= 2 else host
+
+
+def _plain(text: str) -> str:
+    return html.unescape(TAG_RE.sub(" ", text))
+
+
+def _brand_in(brand: str, host: str) -> bool:
+    """Brand match on label boundaries: '.' and '-' delimit, letters and digits do not, so
+    'paypal.com.evil.example' and 'paypal-login.example' hit while 'firstbank.example'
+    (irs), 'backups.example' (ups), 'zoomedphotos.example' and 'purchase-orders.example'
+    (chase) do not. Brands of six or more characters also match inside a label
+    ('microsoftsupport.example'), where a collision with an ordinary word is unlikely."""
+    if re.search(rf"(?<![a-z0-9]){re.escape(brand)}(?![a-z0-9])", host):
+        return True
+    return len(brand) >= 6 and brand in host
+
+
+def _fmt_list(items: list[str], limit: int = 4) -> str:
+    items = list(dict.fromkeys(i.replace("`", "'") for i in items))
+    shown = ", ".join(f"`{i}`" for i in items[:limit])
+    return shown + (f" (+{len(items) - limit} more)" if len(items) > limit else "")
+
+
+def rule_based_cues(subject: str, body: str) -> list[str]:
+    """Heuristic cues as markdown bullets. Deliberately simple regexes: these are reading
+    aids for the analyst, not features of the classifier, so recall matters more than
+    precision and every cue states what it matched."""
+    raw = f"{subject}\n\n{body}"
+    text = _plain(raw)
+    low = text.lower()
+    cues: list[str] = []
+
+    hits = [w for w in URGENCY_WORDS if w in low]
+    if hits:
+        cues.append(f"**Urgency / pressure language:** {_fmt_list(hits)}")
+    hits = [p for p in CREDENTIAL_LURE_PHRASES if p in low]
+    if hits:
+        cues.append(f"**Credential-lure phrasing:** {_fmt_list(hits)}")
+    hits = [g for g in GENERIC_GREETINGS if g in low]
+    if hits:
+        cues.append(f"**Generic greeting (no personal name):** {_fmt_list(hits)}")
+
+    mismatched: list[str] = []
+    for href, inner in ANCHOR_RE.findall(raw):
+        href_host = _host(href)
+        for shown in URL_RE.findall(_plain(inner)):
+            shown_host = _host(shown)
+            if shown_host and href_host and _registrable(shown_host) != _registrable(href_host):
+                mismatched.append(f"{shown_host} -> {href_host}")
+    if mismatched:
+        cues.append(f"**Link text points to a different host than its href:** {_fmt_list(mismatched)}")
+
+    urls = URL_RE.findall(raw) + [h for h, _ in ANCHOR_RE.findall(raw)]
+    hosts = sorted({_host(u) for u in urls if _host(u)})
+    ip_hits = IP_URL_RE.findall(raw)
+    if ip_hits:
+        cues.append(f"**URL with a raw IP address instead of a domain:** {_fmt_list(ip_hits)}")
+    lookalike: list[str] = []
+    for host in hosts:
+        reg = _registrable(host)
+        if reg in KNOWN_BRAND_DOMAINS:
+            continue
+        reg_label = reg.split(".")[0]
+        labels = host.split(".")
+        if any(l.count("-") >= 2 for l in labels):
+            lookalike.append(f"{host} (many hyphens)")
+        deleet = host.translate(LEET)
+        for brand in BRANDS:
+            b = brand.replace(" ", "")
+            in_host = _brand_in(b, host)
+            if not in_host and _brand_in(b, deleet):
+                lookalike.append(f"{host} (digits/symbols standing in for letters in '{brand}')")
+                break
+            if in_host and reg_label != b and not reg.startswith(b + "."):
+                lookalike.append(f"{host} ('{brand}' appears, but the registered domain is {reg})")
+                break
+        if reg in SHORTENERS:
+            lookalike.append(f"{host} (URL shortener hides the destination)")
+    if lookalike:
+        cues.append(f"**Suspicious / lookalike domain:** {_fmt_list(lookalike)}")
+
+    words = [w for w in re.findall(r"[A-Za-z]{3,}", text)]
+    caps = [w for w in words if w.isupper()]
+    if len(words) >= 8 and len(caps) >= 5 and len(caps) / len(words) >= 0.30:
+        cues.append(f"**Heavy ALL-CAPS use:** {len(caps)} of {len(words)} words ({100 * len(caps) / len(words):.0f}%)")
+
+    redirects = [m.group(1) for m in REPLY_REDIRECT_RE.finditer(text)] + REPLY_TO_HEADER_RE.findall(raw)
+    if redirects:
+        cues.append(f"**Asks for a reply to a specific (possibly different) address:** {_fmt_list(redirects)}")
+
+    no_urls = URL_RE.sub(" ", text)  # a URL path like /share/open.htm is a link, not an attachment
+    attachments: list[str] = []
+    for m in ATTACHMENT_RE.finditer(no_urls):
+        if m.group(1).lower() in AMBIGUOUS_ATTACHMENT_EXT:
+            window = no_urls[max(0, m.start() - ATTACHMENT_WINDOW): m.end() + ATTACHMENT_WINDOW]
+            if not ATTACHMENT_CONTEXT_RE.search(window):
+                continue
+        attachments.append(m.group(0).strip("."))
+    if attachments:
+        cues.append(f"**Mentions a risky attachment type (.zip, .htm, .iso, .docm, .exe, ...):** {_fmt_list(attachments)}")
+    return cues
+
+
+# ----------------------------------------------------------------------------- the Gradio function
+def predict(subject: str, body: str) -> tuple[dict[str, float], float, list[tuple[str, str | None]], str]:
+    """Gradio entry point: (class probabilities, prediction time s, saliency highlights, signals markdown).
+
+    The model input is ``subject + "\\n\\n" + body`` exactly as in training (never sender
+    or headers: TREC headers are synthetic). Prediction time covers tokenize + forward +
+    softmax only, matching the instructor's benchmark definition; saliency and rules
+    are timed separately and reported in the signals panel.
+    """
+    # A leading "[SYNTHETIC]" demo tag is UI metadata, not email text: it is stripped so the
+    # classifier (which never saw such a token sequence in training) scores only the email.
+    marked = bool(SYNTHETIC_MARK_RE.match(subject or ""))
+    subject = SYNTHETIC_MARK_RE.sub("", subject or "").strip()
+    body = (body or "").rstrip()
+    text = f"{subject}\n\n{body}"
+    if not text.strip():
+        raise gr.Error("Paste an email subject and/or body first.")
+
+    out = _infer(text)
+    probs: list[float] = out["probs"]
+    p_unwanted: float = out["p_unwanted"]
+    verdict = LABELS[out["verdict"]]
+    label_probs = {name: p for name, p in zip(LABELS, probs)}
+    highlights = saliency_highlights(text, out)
+
+    t0 = timer()
+    cues = rule_based_cues(subject, body)
+    rules_s = timer() - t0
+    class_line = ", ".join(f"{name} {100 * p:.1f}%" for name, p in zip(LABELS, probs))
+    lines = [
+        f"### Model verdict: **{verdict}**",
+        f"**{UNWANTED_LABEL}: {100 * p_unwanted:.1f}%**",
+        f"Per class: {class_line}.",
+        f"The probabilities come from the {N_PARAMS:,}-parameter encoder alone (one softmax over {NUM_LABELS} classes; "
+        f"\"{UNWANTED_LABEL}\" is the sum of the non-legitimate classes, decided at 50%). The red words are the tokens "
+        f"whose gradient x input saliency pushed hardest toward **{verdict}** (top {TOP_K_TOKENS} tokens, merged into words).",
+        "",
+        "### Heuristic cues (rules, not the model)",
+    ]
+    if cues:
+        lines += [f"- {c}" for c in cues]
+    else:
+        lines.append("- _No rule-based cue fired._ Rules are a reading aid; the model can still be right or wrong without them.")
+    if marked:
+        lines.append("- _Leading `[SYNTHETIC]` demo tag removed from the subject before scoring._")
+    trunc = (
+        f"; **input truncated** to the first {out['n_tokens']} of {out['full_len']} tokens (the model never sees the rest)"
+        if out["truncated"] else ""
+    )
+    lines += [
+        "",
+        "### Timing",
+        f"- Prediction (tokenize + forward + softmax): **{out['elapsed_s']:.4f} s** on {out['device']} ({out['dtype']})",
+        f"- Saliency (extra forward + backward): {out['saliency_s']:.4f} s; rules: {rules_s:.4f} s",
+        f"- Tokens: {out['n_tokens']} of max {MAX_LEN}{trunc}",
+    ]
+    return label_probs, round(out["elapsed_s"], 5), highlights, "\n".join(lines)
+
+
+# ----------------------------------------------------------------------------- UI text and examples
+# The static text is derived from the LOADED export (LABELS, NUM_LABELS, N_PARAMS, MAX_LEN) so a
+# two-class fallback export never ships prose promising three classes next to a two-entry Label.
+def _params_human(n: int) -> str:
+    return f"{n / 1e6:.0f}M" if n >= 1_000_000 else f"{n:,}"
+
+
+UNWANTED_NAMES: list[str] = LABELS[LEGIT_INDEX + 1:]
+_CLASS_LIST = ", ".join(f"**{n}**" for n in LABELS)
+_UNWANTED_SUM = " + ".join(f"P({n})" for n in UNWANTED_NAMES)
+_UNWANTED_PICK = (
+    f"the larger of {' / '.join(UNWANTED_NAMES)}" if len(UNWANTED_NAMES) > 1 else f"**{UNWANTED_NAMES[0]}**"
+)
+_THREE_WAY = NUM_LABELS == 3 and UNWANTED_NAMES == DEFAULT_LABELS[3][1:]
+_APPROX_CHARS = f"{round(MAX_LEN * 3.5, -2):,.0f}"  # ~3.5 characters per BPE token on English email
+
+if _THREE_WAY:
+    _WHAT_IT_RETURNS = (
+        "the model estimates the probability that it is **legitimate** business mail, a **phishing** attempt "
+        "(credential, payment or advance-fee lure) or bulk **spam**. The headline roll-up "
+        f"**{UNWANTED_LABEL}** is the sum of the last two."
+    )
+    _PROVENANCE_BULLET = (
+        '- **Spam vs phishing follows corpus provenance, not a per-message review.** "Phishing" rows come from the\n'
+        '  Nazario phishing archive, the Nigerian-fraud (advance-fee) archive and BITS synthetic lures; "Spam" rows are\n'
+        "  the SpamAssassin and Enron-Spam spam folders. The TREC-05/07 and CEAS-08 positives never separated the two\n"
+        '  and were trained only as "unwanted". Expect the phishing/spam boundary to be softer than legitimate/unwanted.\n'
+    )
+    _SALIENCY_TARGET = "toward the combined phishing + spam mass"
+else:
+    _WHAT_IT_RETURNS = (
+        f"the model returns one probability per class ({_CLASS_LIST}). The headline roll-up "
+        f"**{UNWANTED_LABEL}** is {_UNWANTED_SUM}: this export does not separate phishing from spam."
+    )
+    _PROVENANCE_BULLET = (
+        "- This export has a single unwanted class: phishing and bulk spam are not told apart.\n"
+    )
+    _SALIENCY_TARGET = f"toward {_UNWANTED_SUM}"
+
+DESCRIPTION = f"""
+Paste an email **subject** and **body** and {_WHAT_IT_RETURNS}
+
+The classifier is a **{_params_human(N_PARAMS)}-parameter BERT-style encoder written from scratch in plain PyTorch**
+(no `transformers`, no pretrained weights): masked-language-model pretraining on deduplicated Enron email, then
+fine-tuning on labelled phishing/spam corpora. It is a **defensive** tool for triage and training, built as a
+course project by Build IT Systems (BITS).
+
+**Do not paste secrets, passwords, or confidential client content.** The examples below are **synthetic**,
+written for this demo; none is a real message, and every domain in them is a reserved `.example` name.
+"""
+ARTICLE = f"""
+### How to read the result
+- **Prediction** is one softmax over {NUM_LABELS} classes: {_CLASS_LIST}. The Signals panel
+  adds **{UNWANTED_LABEL}** = {_UNWANTED_SUM}, decided at 50%; the verdict is {LABELS[LEGIT_INDEX]} below
+  that line and otherwise {_UNWANTED_PICK}. The roll-up is the number to compare with the binary
+  runs in the project report.
+- **Prediction Time (s)** is tokenize + forward pass + softmax for this one email, measured with
+  `timeit.default_timer` exactly like the course reference app. The instructor's benchmark is 0.074 s per sample.
+- **Tokens the model weighed most** are gradient x input saliency scores over the model's own token embeddings:
+  red words pushed the decision toward the verdict shown (for an unwanted verdict, {_SALIENCY_TARGET}).
+  They explain *this model*, not phishing in general.
+- **Signals** lists simple regex heuristics (urgency words, credential lures, host mismatches, lookalike domains,
+  raw-IP links, ALL-CAPS, reply-to redirection, risky attachment names). They are shown for context only and do
+  **not** feed the probability.
+
+### Limitations
+{_PROVENANCE_BULLET}- Trained on 1995-2022 English email, mostly 1998-2008 (Enron, TREC, CEAS, SpamAssassin). Modern lures (MFA
+  fatigue, QR codes, Teams/Slack phishing) and non-English mail are out of distribution.
+- The model reads subject and body text only: no sender, headers, SPF/DKIM, attachments or link reputation.
+- Inputs are truncated to {MAX_LEN} tokens (roughly {_APPROX_CHARS} characters); long emails are judged on their opening.
+- Length is a known confound in the training data (legitimate mail is on average longer); the error analysis in
+  the project report checks for it.
+- A probability is not a verdict. Use it to prioritise human review, never to auto-delete mail.
+
+### Links
+- Model card and weights: *[private model repo link placeholder]*
+- Training code, run log and metrics: *[project repository link placeholder]*
+"""
+PLACEHOLDER = (
+    "Paste the email body here. Plain text or HTML both work; links such as <a href=...> are scanned for "
+    "host mismatches. Nothing you paste is stored (flagging is disabled)."
+)
+
+EXAMPLES: list[list[str]] = [
+    [
+        "Action Required: Your Microsoft 365 password expires in 24 hours",
+        "Dear user,\n\nYour Office 365 password will expire within 24 hours. To avoid losing access to your mailbox, "
+        "verify your account immediately by signing in at:\n\n"
+        '<a href="http://login.micr0soft-365-verify.example/reset?u=8f3a9c">https://login.microsoftonline.com</a>\n\n'
+        "Failure to act will result in permanent suspension of your account.\n\nIT Security Team\nOffice 365 Support",
+    ],
+    [
+        "Re: Invoice 4471 - updated banking details",
+        "Hi Jordan,\n\nQuick one before I board. Our finance team moved banks last week, so please send the payment "
+        "for invoice 4471 (USD 48,200) to the new account below rather than the one on the PO. Treat this as time "
+        "sensitive; the supplier is holding the shipment until funds clear.\n\nBank: First Meridian\nAccount: "
+        "0031-7742-19\nSWIFT: FMRDUS33\n\nI am in meetings all day so email me at ceo.office.priv@fastmail-relay.example "
+        "if anything is unclear rather than calling.\n\nThanks,\nMichael\nChief Executive Officer",
+    ],
+    [
+        "A document has been shared with you: Q3_Payroll_Review.docm",
+        "Hello,\n\nA colleague shared a file with you via SecureShare.\n\nFile: Q3_Payroll_Review.docm (1.2 MB)\n"
+        "Expires: today\n\nOpen document: http://185.212.47.19/share/open.htm?id=771\n\nIf you cannot open the "
+        "link, download the attached Q3_Payroll_Review.zip and enable content when prompted.\n\nThis is an automated "
+        "message. Please do not reply.",
+    ],
+    [
+        "FINAL HOURS: 70% OFF luxury watches + FREE shipping!!!",
+        "CONGRATULATIONS! You have been SELECTED for our EXCLUSIVE VIP SALE.\n\n*** 70% OFF all designer watches, "
+        "sunglasses and handbags ***\n*** FREE worldwide shipping - TODAY ONLY ***\n*** Buy 2, get 1 FREE ***\n\n"
+        "Thousands of satisfied customers can't be wrong! Shop now: http://bit.ly/vip-watch-deals-2026\n\n"
+        "Don't miss out - this offer expires at midnight!\n\nTo unsubscribe, reply with REMOVE in the subject line.",
+    ],
+    [
+        "Agenda for Thursday's project sync",
+        "Hi all,\n\nAttaching the agenda for Thursday's 10:00 sync. Main items:\n\n1. Status of the firewall "
+        "migration (Dana)\n2. Q4 hardware refresh budget (Priya)\n3. Client onboarding checklist review\n\nIf you "
+        "want anything added, reply by Wednesday noon. Dial-in details are unchanged from last week.\n\nThanks,\n"
+        "Nathan",
+    ],
+    [
+        "Re: Contract renewal - redlines attached",
+        "Hi Sarah,\n\nThanks for turning the redlines around so quickly. We accepted the changes to clauses 4.2 and "
+        "7.1 and left a comment on the termination notice period; 60 days works for us but legal prefers 90. "
+        "Can we close this out on the call Friday?\n\nAlso, the summary report for September is on the shared "
+        "drive under Reports/2026-09.\n\nBest,\nMarcus",
+    ],
+]
+# Shown in the Examples table instead of the raw rows. The "synthetic" marker lives here, in
+# the UI only, so the model input is exactly the email text (a "[SYNTHETIC]" token sequence
+# in the subject would be out of distribution and perturb the demo predictions).
+EXAMPLE_LABELS: list[str] = [
+    "Synthetic lure: Microsoft 365 password-expiry credential phish",
+    "Synthetic lure: CEO asks to pay an invoice to new bank details (BEC)",
+    "Synthetic lure: shared .docm file, raw-IP link, .zip fallback",
+    "Synthetic spam: shouting discount blast with a shortener link",
+    "Legitimate: agenda for a project sync",
+    "Legitimate: contract redlines follow-up",
+]
+
+
+def build_demo() -> gr.Interface:
+    return gr.Interface(
+        fn=predict,
+        inputs=[
+            gr.Textbox(label="Subject", lines=1, placeholder="Email subject line"),
+            gr.Textbox(label="Email body", lines=14, placeholder=PLACEHOLDER),
+        ],
+        outputs=[
+            gr.Label(num_top_classes=NUM_LABELS, label="Prediction"),
+            gr.Number(label="Prediction Time (s)"),
+            gr.HighlightedText(
+                label="Tokens the model weighed most",
+                color_map={"high": "#d62728", "medium": "#ff9896"},
+                combine_adjacent=True,
+                show_legend=False,
+            ),
+            gr.Markdown(label="Signals"),
+        ],
+        title=TITLE,
+        description=DESCRIPTION,
+        article=ARTICLE,
+        examples=EXAMPLES,
+        example_labels=EXAMPLE_LABELS,
+        cache_examples=False,  # Spaces default is True; caching would run the GPU function at build time
+        flagging_mode="never",
+        analytics_enabled=False,
+        api_name="predict",
+    )
+
+
+# ----------------------------------------------------------------------------- development entry points
+SELFTEST_EMAILS: list[tuple[str, str]] = [EXAMPLES[0][:2], EXAMPLES[3][:2], EXAMPLES[4][:2]]  # lure, spam, legitimate
+
+
+def _set_device(device: str) -> None:
+    global MODEL, DEVICE
+    DEVICE = device
+    MODEL = place_model(MODEL, device)
+
+
+def selftest() -> int:
+    devices = ["cpu"] + (["cuda"] if torch.cuda.is_available() and not ZERO_GPU else [])
+    for device in devices:
+        _set_device(device)
+        LOG.info("selftest on %s", device)
+        for subject, body in SELFTEST_EMAILS:
+            probs, secs, highlights, details = predict(subject, body)
+            painted = [(t, l) for t, l in highlights if l]
+            print(f"\n=== [{device}] {subject}")
+            print(f"probs={ {k: round(v, 4) for k, v in probs.items()} } prediction_time_s={secs}")
+            print(f"highlighted words ({len(painted)}): {painted}")
+            print(details)
+            assert list(probs) == LABELS and len(probs) == NUM_LABELS, "one probability per class, in index order"
+            assert abs(sum(probs.values()) - 1.0) < 1e-5
+            p_unwanted = sum(p for name, p in probs.items() if name != LABELS[LEGIT_INDEX])
+            assert f"{UNWANTED_LABEL}: {100 * p_unwanted:.1f}%" in details, "headline must report the unwanted roll-up"
+            verdict_idx, _ = verdict_from_probs([probs[n] for n in LABELS])
+            assert f"**{LABELS[verdict_idx]}**" in details, "verdict wording must follow verdict_from_probs"
+            expected = f"{SYNTHETIC_MARK_RE.sub('', subject).strip()}\n\n{body.rstrip()}"
+            assert "".join(t for t, _ in highlights) == expected, "highlights must cover the model input text"
+    # The static UI text must describe the export that is actually loaded.
+    assert all(name in ARTICLE for name in LABELS), "ARTICLE must name every loaded class"
+    assert f"one softmax over {NUM_LABELS} classes" in ARTICLE and f"{MAX_LEN} tokens" in ARTICLE
+    assert UNWANTED_LABEL in DESCRIPTION and _params_human(N_PARAMS) in DESCRIPTION
+    _set_device("cuda" if torch.cuda.is_available() else "cpu")
+    print("\nselftest OK")
+    return 0
+
+
+def serve_for(seconds: float) -> int:
+    import urllib.request
+
+    demo = build_demo()
+    _app, local_url, _share = demo.launch(prevent_thread_lock=True, quiet=True)
+    print(f"serving at {local_url} for {seconds:.0f}s")
+    status = -1
+    for _ in range(20):
+        try:
+            with urllib.request.urlopen(local_url, timeout=5) as resp:
+                status = resp.status
+                page = resp.read(4096).decode("utf-8", "replace")
+            break
+        except Exception:  # server still starting
+            time.sleep(0.5)
+    print(f"GET {local_url} -> HTTP {status}; html head contains '<html': {'<html' in page.lower() if status == 200 else False}")
+    time.sleep(seconds)
+    demo.close()
+    return 0 if status == 200 else 1
+
+
+if not ZERO_GPU and __name__ == "__main__":
+    # Warm the kernels (CUDA context / oneDNN) so the first user click is not the slow one.
+    # Skipped under ZeroGPU: CUDA work is only legal inside @spaces.GPU functions there.
+    with torch.inference_mode(), _autocast(DEVICE):
+        MODEL(torch.full((1, 16), 5, dtype=torch.int64, device=DEVICE), torch.ones(1, 16, dtype=torch.bool, device=DEVICE))
+    LOG.info("warm-up forward done on %s", DEVICE)
+
+if __name__ == "__main__":
+    assert ARGS is not None
+    if ARGS.selftest:
+        raise SystemExit(selftest())
+    if ARGS.serve_seconds is not None:
+        raise SystemExit(serve_for(ARGS.serve_seconds))
+    demo = build_demo()
+    demo.launch()
+else:
+    demo = build_demo()
diff --git a/presentation/app/model.py b/presentation/app/model.py
new file mode 100644
index 0000000000000000000000000000000000000000..cc07f2f4c77390f7a9d0e48a3c46f7ae1c543e63
--- /dev/null
+++ b/presentation/app/model.py
@@ -0,0 +1,524 @@
+"""From-scratch pre-LayerNorm BERT-style encoder for the BITS Phishing Detector.
+
+Plain PyTorch only (torch / math / dataclasses / typing) so this file can be copied
+verbatim to app/model.py for the Hugging Face Space. No pretrained weights, no
+``transformers``.
+
+Architecture (docs/research/research_model-math.md sec. 1 and 5):
+
+* token + learned absolute position embeddings -> LayerNorm -> dropout (no segment
+  embeddings: the classifier sees a single "subject\\n\\nbody" segment).
+* ``n_layers`` pre-LN blocks::
+
+      x = x + Dropout(OutProj(SDPA(QKV(LN1(x)))))
+      x = x + Dropout(FFN_down(GELU(FFN_up(LN2(x)))))
+
+  Pre-LN is chosen over BERT's post-LN because a 24-layer stack trained from
+  scratch at batch 256 is far more tolerant of the learning rate and warmup under
+  pre-LN (Xiong et al. 2020), and we have no pretrained weights to fall back on.
+* a final LayerNorm (pre-LN needs it: the residual stream is otherwise un-normalised).
+* attention through ``torch.nn.functional.scaled_dot_product_attention`` so the
+  fused Flash / memory-efficient kernels run and the L x L score matrix is never
+  stored for backward. No mask -> ``attn_mask=None`` (Flash-eligible, the packed
+  pretraining path); a ``BoolTensor[B, L]`` key-padding mask (True = keep) ->
+  ``[B, 1, 1, L]`` (memory-efficient backend, the fine-tune / inference path).
+
+Default config (24L / d1024 / 16 heads / FFN 4096 / V 32768 / L 512) has exactly
+337,476,608 parameters with the tied MLM head and 337,443,842 as the classifier.
+"""
+
+import math
+from dataclasses import asdict, dataclass, fields
+from typing import Any, Mapping
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from torch import Tensor
+from torch.utils.checkpoint import checkpoint
+
+
+@dataclass
+class EncoderConfig:
+    vocab_size: int = 32768
+    max_seq_len: int = 512
+    d_model: int = 1024
+    n_layers: int = 24
+    n_heads: int = 16
+    d_ff: int = 4096
+    dropout: float = 0.1
+    attention_dropout: float = 0.1
+    # 1e-5 rather than BERT's 1e-12. Under autocast LayerNorm runs in fp32 so both
+    # behave identically in training; 1e-5 is the safer value if the deployed model
+    # is ever run in pure bf16 (8-bit mantissa: var + 1e-12 == var), and it is the
+    # value every modern pre-LN encoder (RoBERTa-style, GPT-2) uses.
+    layer_norm_eps: float = 1e-5
+    pad_token_id: int = 0
+    tie_mlm_head: bool = True
+    grad_checkpointing: bool = False
+    num_labels: int = 2
+
+    def __post_init__(self) -> None:
+        if self.d_model % self.n_heads:
+            raise ValueError(f"d_model={self.d_model} not divisible by n_heads={self.n_heads}")
+        if not 0 <= self.pad_token_id < self.vocab_size:
+            raise ValueError("pad_token_id outside the vocabulary")
+
+    @property
+    def head_dim(self) -> int:
+        return self.d_model // self.n_heads
+
+    @classmethod
+    def from_dict(cls, d: Mapping[str, Any]) -> "EncoderConfig":
+        """Build from the config.yaml ``model`` section.
+
+        Keys that describe choices this file implements only one way (``norm``,
+        ``activation``) are validated rather than silently ignored; any other
+        unknown key is dropped.
+        """
+        norm = d.get("norm", "pre_ln")
+        if norm != "pre_ln":
+            raise ValueError(f"only norm=pre_ln is implemented, got {norm!r}")
+        act = d.get("activation", "gelu")
+        if act != "gelu":
+            raise ValueError(f"only activation=gelu is implemented, got {act!r}")
+        known = {f.name for f in fields(cls)}
+        return cls(**{k: v for k, v in d.items() if k in known})
+
+    def to_dict(self) -> dict[str, Any]:
+        return asdict(self)
+
+
+# --------------------------------------------------------------------------- init
+def _init_module(module: nn.Module) -> None:
+    """BERT init: N(0, 0.02) weights, zero biases, identity LayerNorm."""
+    if isinstance(module, nn.Linear):
+        nn.init.normal_(module.weight, mean=0.0, std=0.02)
+        if module.bias is not None:
+            nn.init.zeros_(module.bias)
+    elif isinstance(module, nn.Embedding):
+        nn.init.normal_(module.weight, mean=0.0, std=0.02)
+        if module.padding_idx is not None:
+            with torch.no_grad():
+                module.weight[module.padding_idx].zero_()
+    elif isinstance(module, nn.LayerNorm):
+        nn.init.ones_(module.weight)
+        nn.init.zeros_(module.bias)
+
+
+def _zero_pad_row_grad(pad_token_id: int):
+    """Post-accumulate hook keeping the [PAD] embedding row exactly zero through training.
+
+    ``nn.Embedding(padding_idx)`` routes no gradient into that row, but with the tied MLM
+    decoder the output matmul does (observed grad norm ~0.06 after one backward), so
+    the row would drift from zero on the first optimizer step. Zeroing ``.grad[pad]``
+    after accumulation is enough: AdamW then has zero moments for the row and decoupled
+    weight decay leaves a zero weight at zero.
+    """
+    def hook(param: Tensor) -> None:
+        if param.grad is not None:
+            param.grad[pad_token_id].zero_()
+    return hook
+
+
+def _expand_key_padding_mask(attention_mask: Tensor | None) -> Tensor | None:
+    """``BoolTensor[B, L]`` (True = keep) -> ``[B, 1, 1, L]`` broadcast over heads and queries."""
+    if attention_mask is None:
+        return None
+    if attention_mask.dtype is not torch.bool:
+        attention_mask = attention_mask.bool()
+    return attention_mask[:, None, None, :]
+
+
+# ------------------------------------------------------------------------ modules
+class Embeddings(nn.Module):
+    """Token + learned absolute position embeddings -> LayerNorm -> dropout.
+
+    The [PAD] row of the token table is zero at init and stays zero while training
+    (see ``_zero_pad_row_grad``). Tensor hooks are not carried by ``state_dict`` or
+    ``copy.deepcopy``: a deep-copied module must not be trained, a module rebuilt from
+    its config and ``load_state_dict`` is fine.
+    """
+
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.tok = nn.Embedding(cfg.vocab_size, cfg.d_model, padding_idx=cfg.pad_token_id)
+        self.pos = nn.Embedding(cfg.max_seq_len, cfg.d_model)
+        self.norm = nn.LayerNorm(cfg.d_model, eps=cfg.layer_norm_eps)
+        self.dropout = nn.Dropout(cfg.dropout)
+        self.register_buffer("position_ids", torch.arange(cfg.max_seq_len), persistent=False)
+        self.tok.weight.register_post_accumulate_grad_hook(_zero_pad_row_grad(cfg.pad_token_id))
+
+    def forward(self, input_ids: Tensor) -> Tensor:
+        L = input_ids.shape[1]
+        if L > self.pos.num_embeddings:
+            raise ValueError(f"sequence length {L} > max_seq_len {self.pos.num_embeddings}")
+        x = self.tok(input_ids) + self.pos(self.position_ids[:L])
+        return self.dropout(self.norm(x))
+
+
+class EncoderLayer(nn.Module):
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.n_heads = cfg.n_heads
+        self.head_dim = cfg.head_dim
+        self.attention_dropout = cfg.attention_dropout
+        self.ln1 = nn.LayerNorm(cfg.d_model, eps=cfg.layer_norm_eps)
+        self.qkv = nn.Linear(cfg.d_model, 3 * cfg.d_model)
+        self.out_proj = nn.Linear(cfg.d_model, cfg.d_model)
+        self.ln2 = nn.LayerNorm(cfg.d_model, eps=cfg.layer_norm_eps)
+        self.ffn_up = nn.Linear(cfg.d_model, cfg.d_ff)
+        self.ffn_down = nn.Linear(cfg.d_ff, cfg.d_model)
+        self.dropout = nn.Dropout(cfg.dropout)
+
+    def _qkv(self, x: Tensor) -> tuple[Tensor, Tensor, Tensor]:
+        B, L, _ = x.shape
+        qkv = self.qkv(x).view(B, L, 3, self.n_heads, self.head_dim)
+        q, k, v = qkv.unbind(dim=2)
+        return q.transpose(1, 2), k.transpose(1, 2), v.transpose(1, 2)  # [B, h, L, dh]
+
+    def forward(self, x: Tensor, attn_mask: Tensor | None = None) -> Tensor:
+        q, k, v = self._qkv(self.ln1(x))
+        o = F.scaled_dot_product_attention(
+            q, k, v,
+            attn_mask=attn_mask,
+            dropout_p=self.attention_dropout if self.training else 0.0,
+        )
+        o = o.transpose(1, 2).reshape(x.shape)
+        x = x + self.dropout(self.out_proj(o))
+        x = x + self.dropout(self.ffn_down(F.gelu(self.ffn_up(self.ln2(x)))))
+        return x
+
+    @torch.no_grad()
+    def attention_probs(self, x: Tensor, attn_mask: Tensor | None = None) -> Tensor:
+        """Explicit softmax(QK^T / sqrt(dh)) for inspection. Off the hot path.
+
+        fp32, no dropout: this is the quantity the app shows as a signal, not the
+        stochastic one used in training. Returns ``[B, h, L, L]``.
+        """
+        q, k, _ = self._qkv(self.ln1(x))
+        scores = (q.float() @ k.float().transpose(-1, -2)) / math.sqrt(self.head_dim)
+        if attn_mask is not None:
+            scores = scores.masked_fill(~attn_mask, float("-inf"))
+        return scores.softmax(dim=-1)
+
+
+class Encoder(nn.Module):
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.config = cfg
+        self.embeddings = Embeddings(cfg)
+        self.layers = nn.ModuleList(EncoderLayer(cfg) for _ in range(cfg.n_layers))
+        self.final_norm = nn.LayerNorm(cfg.d_model, eps=cfg.layer_norm_eps)
+        self.apply(_init_module)
+        self._scale_residual_projections()
+
+    def _scale_residual_projections(self) -> None:
+        """GPT-2 style 1/sqrt(2 * n_layers) on the two projections that write into the
+        residual stream (OutProj, FFN_down).
+
+        Each of the 2 * n_layers residual branches adds an N(0, 0.02^2)-scaled
+        contribution; without the rescale the residual variance grows linearly with
+        depth, so the final-norm input and the early-training gradient magnitude
+        would be ~sqrt(48) ~ 7x larger for 24 layers than for one. Keeping the stream
+        O(1) at init is what lets a from-scratch 24-layer pre-LN model take a 1.5e-4
+        peak LR with a short warmup instead of diverging in the first few hundred steps.
+        """
+        scale = 1.0 / math.sqrt(2 * self.config.n_layers)
+        with torch.no_grad():
+            for layer in self.layers:
+                layer.out_proj.weight.mul_(scale)
+                layer.ffn_down.weight.mul_(scale)
+
+    def forward(
+        self,
+        input_ids: Tensor,
+        attention_mask: Tensor | None = None,
+        return_last_attn: bool = False,
+    ) -> Tensor | tuple[Tensor, Tensor]:
+        mask = _expand_key_padding_mask(attention_mask)
+        x = self.embeddings(input_ids)
+        use_ckpt = self.config.grad_checkpointing and self.training and torch.is_grad_enabled()
+        last_attn: Tensor | None = None
+        n = len(self.layers)
+        for i, layer in enumerate(self.layers):
+            if return_last_attn and i == n - 1:
+                last_attn = layer.attention_probs(x, mask)
+            if use_ckpt:
+                x = checkpoint(layer, x, mask, use_reentrant=False)
+            else:
+                x = layer(x, mask)
+        x = self.final_norm(x)
+        if return_last_attn:
+            assert last_attn is not None
+            return x, last_attn
+        return x
+
+
+class MLMHead(nn.Module):
+    """Linear(d, d) -> GELU -> LayerNorm -> decoder Linear(d, V) tied to the token embedding.
+
+    Tying shares the Parameter object, so ``state_dict()`` carries two keys over one
+    storage (``encoder.embeddings.tok.weight`` and ``head.decoder.weight``) and
+    ``safetensors.torch.save_file`` refuses it ("Some tensors share memory"). Write
+    checkpoints from ``EncoderForMLM.state_dict_for_safetensors()`` (or with
+    ``safetensors.torch.save_model``, which drops the same alias) and read them back
+    with ``EncoderForMLM.load_tied_state_dict``.
+    """
+
+    def __init__(self, cfg: EncoderConfig, token_embedding: nn.Embedding | None) -> None:
+        super().__init__()
+        self.dense = nn.Linear(cfg.d_model, cfg.d_model)
+        self.norm = nn.LayerNorm(cfg.d_model, eps=cfg.layer_norm_eps)
+        self.decoder = nn.Linear(cfg.d_model, cfg.vocab_size, bias=False)
+        # Own bias so the tied matrix is not forced to also fit unigram frequencies.
+        self.bias = nn.Parameter(torch.zeros(cfg.vocab_size))
+        self.apply(_init_module)
+        if token_embedding is not None:
+            # Same Parameter object: one set of weights, one gradient accumulator.
+            # Tying saves 33.5M parameters and (BERT/RoBERTa) regularises the
+            # output distribution toward the input geometry.
+            self.decoder.weight = token_embedding.weight
+
+    def forward(self, hidden: Tensor) -> Tensor:
+        h = self.norm(F.gelu(self.dense(hidden)))
+        return self.decoder(h) + self.bias
+
+
+class EncoderForMLM(nn.Module):
+    """Encoder + MLM head for pretraining. Checkpoints: see ``state_dict_for_safetensors``."""
+
+    TIED_KEY = "head.decoder.weight"
+    EMBEDDING_KEY = "encoder.embeddings.tok.weight"
+
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.config = cfg
+        self.encoder = Encoder(cfg)
+        self.head = MLMHead(cfg, self.encoder.embeddings.tok if cfg.tie_mlm_head else None)
+
+    def state_dict_for_safetensors(self) -> dict[str, Tensor]:
+        """``state_dict()`` minus the tied decoder's alias key.
+
+        With ``tie_mlm_head`` the decoder weight IS the token embedding, and safetensors
+        rejects two keys over one storage, so the pretraining trainer must save this
+        dict (``safetensors.torch.save_file(model.state_dict_for_safetensors(), path)``)
+        rather than the raw ``state_dict()``. ``from_pretrained_encoder`` reads only the
+        ``encoder.*`` keys and accepts either form.
+        """
+        sd = self.state_dict()
+        if self.config.tie_mlm_head:
+            sd.pop(self.TIED_KEY)
+        return sd
+
+    def load_tied_state_dict(self, state_dict: Mapping[str, Tensor], strict: bool = True) -> Any:
+        """``load_state_dict`` that accepts the dedup'd dict from ``state_dict_for_safetensors``
+        / ``save_model``: the dropped alias is re-pointed at the embedding tensor before
+        loading, so strict mode still reports every genuine mismatch."""
+        sd = dict(state_dict)
+        if self.config.tie_mlm_head and self.TIED_KEY not in sd and self.EMBEDDING_KEY in sd:
+            sd[self.TIED_KEY] = sd[self.EMBEDDING_KEY]
+        return self.load_state_dict(sd, strict=strict)
+
+    def forward(
+        self,
+        input_ids: Tensor,
+        attention_mask: Tensor | None = None,
+        masked_positions: Tensor | None = None,
+    ) -> Tensor:
+        """Logits over the vocabulary.
+
+        With ``masked_positions`` (``BoolTensor[B, L]``) the hidden states are
+        gathered BEFORE the head, so logits are ``[N_masked, V]`` and the 85% of
+        positions that carry no loss never materialise a V-wide row (saves ~136 MiB
+        per 512-token sample at peak; research_model-math.md sec. 2).
+        Without it, ``[B, L, V]``.
+        """
+        hidden = self.encoder(input_ids, attention_mask)
+        if masked_positions is not None:
+            hidden = hidden[masked_positions]
+        return self.head(hidden)
+
+
+class ClassificationHead(nn.Module):
+    """CLS -> dropout -> Linear(d, d) -> tanh -> dropout -> Linear(d, num_labels)."""
+
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.dropout = nn.Dropout(cfg.dropout)
+        self.dense = nn.Linear(cfg.d_model, cfg.d_model)
+        self.out = nn.Linear(cfg.d_model, cfg.num_labels)
+        self.apply(_init_module)
+
+    def forward(self, cls_hidden: Tensor) -> Tensor:
+        h = torch.tanh(self.dense(self.dropout(cls_hidden)))
+        return self.out(self.dropout(h))
+
+
+class EncoderForClassification(nn.Module):
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.config = cfg
+        self.encoder = Encoder(cfg)
+        self.classifier = ClassificationHead(cfg)
+
+    def forward(
+        self,
+        input_ids: Tensor,
+        attention_mask: Tensor | None = None,
+        return_last_attn: bool = False,
+    ) -> Tensor | tuple[Tensor, Tensor]:
+        out = self.encoder(input_ids, attention_mask, return_last_attn=return_last_attn)
+        if return_last_attn:
+            hidden, last_attn = out
+            return self.classifier(hidden[:, 0]), last_attn
+        return self.classifier(out[:, 0])
+
+    @classmethod
+    def from_pretrained_encoder(
+        cls,
+        mlm: "EncoderForMLM | Mapping[str, Tensor]",
+        cfg: EncoderConfig | None = None,
+        num_labels: int | None = None,
+    ) -> "EncoderForClassification":
+        """Classifier whose encoder is initialised from a pretrained ``EncoderForMLM``
+        (module or its ``state_dict``). The MLM head is dropped; the classification
+        head keeps its fresh init. Strict key matching: any missing/unexpected encoder
+        key raises, because a silently half-loaded encoder would train but be garbage.
+        """
+        if isinstance(mlm, EncoderForMLM):
+            cfg = cfg or mlm.config
+            state = mlm.state_dict()
+        else:
+            if cfg is None:
+                raise ValueError("cfg is required when loading from a state_dict")
+            state = mlm
+        if num_labels is not None and num_labels != cfg.num_labels:
+            cfg = EncoderConfig(**{**cfg.to_dict(), "num_labels": num_labels})
+        model = cls(cfg)
+        load_encoder_weights(model.encoder, state, prefix="encoder.", strict=True)
+        return model
+
+
+def load_encoder_weights(
+    encoder: Encoder,
+    state_dict: Mapping[str, Tensor],
+    prefix: str = "encoder.",
+    strict: bool = True,
+) -> tuple[list[str], list[str]]:
+    """Load ``Encoder`` weights from a wrapper's state_dict (keys ``encoder.*``).
+
+    Returns ``(missing_keys, unexpected_keys)`` relative to the encoder. Keys outside
+    ``prefix`` (``head.*``, ``classifier.*``) are ignored, not reported. With
+    ``strict`` any mismatch raises after reporting the exact keys.
+    """
+    sub = {k[len(prefix):]: v for k, v in state_dict.items() if k.startswith(prefix)}
+    result = encoder.load_state_dict(sub, strict=False)
+    missing, unexpected = list(result.missing_keys), list(result.unexpected_keys)
+    if strict and (missing or unexpected):
+        raise RuntimeError(
+            f"encoder weights do not match: missing={missing[:8]}{'...' if len(missing) > 8 else ''} "
+            f"unexpected={unexpected[:8]}{'...' if len(unexpected) > 8 else ''}"
+        )
+    return missing, unexpected
+
+
+def count_parameters(module: nn.Module) -> dict[str, Any]:
+    """Unique-parameter counts (a tied weight is counted once, under the encoder's
+    embedding where ``named_parameters`` first sees it).
+
+    Breakdown keys: ``embeddings`` (token + position + embedding LN),
+    ``encoder_layers``, ``final_norm``, ``head`` (MLM head or classifier).
+    """
+    total = sum(p.numel() for p in module.parameters())
+    trainable = sum(p.numel() for p in module.parameters() if p.requires_grad)
+    breakdown = {"embeddings": 0, "encoder_layers": 0, "final_norm": 0, "head": 0}
+    for name, p in module.named_parameters():
+        parts = name.split(".")
+        if "embeddings" in parts:
+            breakdown["embeddings"] += p.numel()
+        elif "layers" in parts:
+            breakdown["encoder_layers"] += p.numel()
+        elif "final_norm" in parts:
+            breakdown["final_norm"] += p.numel()
+        else:
+            breakdown["head"] += p.numel()
+    return {"total": total, "trainable": trainable, **breakdown}
+
+
+def _format_count_table(title: str, counts: Mapping[str, int]) -> str:
+    rows = [f"{title}"]
+    for k in ("embeddings", "encoder_layers", "final_norm", "head", "total", "trainable"):
+        rows.append(f"  {k:<15} {counts[k]:>15,}")
+    return "\n".join(rows)
+
+
+if __name__ == "__main__":
+    import sys
+    from pathlib import Path
+
+    import yaml  # only here: the module body must stay torch-only for app/model.py
+
+    cfg_path = Path(__file__).resolve().parent.parent / "config.yaml"
+    with open(cfg_path) as fh:
+        model_cfg = EncoderConfig.from_dict(yaml.safe_load(fh)["model"])
+    print(f"config: {model_cfg.to_dict()}")
+
+    mlm = EncoderForMLM(model_cfg)
+    mlm_counts = count_parameters(mlm)
+    print(_format_count_table("EncoderForMLM (tied head)", mlm_counts))
+    assert mlm.head.decoder.weight is mlm.encoder.embeddings.tok.weight, "MLM head is not tied"
+
+    cls = EncoderForClassification.from_pretrained_encoder(mlm)
+    cls_counts = count_parameters(cls)
+    print(_format_count_table("EncoderForClassification", cls_counts))
+
+    is_default = (
+        model_cfg.n_layers, model_cfg.d_model, model_cfg.n_heads, model_cfg.d_ff,
+        model_cfg.vocab_size, model_cfg.max_seq_len, model_cfg.tie_mlm_head, model_cfg.num_labels,
+    ) == (24, 1024, 16, 4096, 32768, 512, True, 2)
+    if is_default:
+        # research_model-math.md sec. 1: per layer 12,596,224; encoder 302,311,424;
+        # embeddings 34,080,768; tied MLM head 1,084,416; classifier head 1,051,650.
+        assert mlm_counts["total"] == 337_476_608, mlm_counts["total"]
+        assert cls_counts["total"] == 337_443_842, cls_counts["total"]
+        assert mlm_counts["encoder_layers"] == 302_309_376, mlm_counts["encoder_layers"]
+        assert mlm_counts["embeddings"] == 34_080_768, mlm_counts["embeddings"]
+        assert mlm_counts["head"] == 1_084_416, mlm_counts["head"]
+        assert cls_counts["head"] == 1_051_650, cls_counts["head"]
+        print("parameter counts match research_model-math.md sec. 1")
+    else:
+        print("non-default config: skipping the 337,476,608 / 337,443,842 assertion")
+
+    # Checkpoint format: the tied alias is dropped for safetensors and restored on load.
+    sd = mlm.state_dict_for_safetensors()
+    assert EncoderForMLM.TIED_KEY not in sd and len(sd) == len(mlm.state_dict()) - 1
+    mlm.load_tied_state_dict(sd, strict=True)
+    EncoderForClassification.from_pretrained_encoder(sd, model_cfg)
+    print(f"safetensors-safe state_dict: {len(sd)} tensors, strict round-trip OK")
+
+    # The [PAD] embedding row must receive no gradient despite the tied decoder.
+    ids = torch.randint(5, model_cfg.vocab_size, (2, 16))
+    mlm.train()
+    mlm(ids).float().mean().backward()
+    pad_grad = mlm.encoder.embeddings.tok.weight.grad[model_cfg.pad_token_id]
+    assert float(pad_grad.abs().max()) == 0.0, "pad row received gradient"
+    assert float(mlm.encoder.embeddings.tok.weight.grad[5:].abs().max()) > 0.0
+    mlm.zero_grad(set_to_none=True)
+    print("pad-row gradient is zero under the tied decoder")
+
+    # Round-trip check of the shared contract on a tiny CPU batch.
+    mlm.eval()
+    mask = torch.ones(2, 16, dtype=torch.bool)
+    mask[1, 10:] = False
+    with torch.inference_mode():
+        full = mlm(ids, mask)
+        pos = torch.zeros(2, 16, dtype=torch.bool)
+        pos[0, 3] = pos[1, 7] = True
+        gathered = mlm(ids, mask, masked_positions=pos)
+        logits, attn = cls(ids, mask, return_last_attn=True)
+    assert full.shape == (2, 16, model_cfg.vocab_size) and gathered.shape == (2, model_cfg.vocab_size)
+    assert logits.shape == (2, model_cfg.num_labels) and attn.shape == (2, model_cfg.n_heads, 16, 16)
+    assert torch.allclose(attn[1, :, :, 10:], torch.zeros(()), atol=0), "padded keys received attention"
+    print("forward shapes OK:", tuple(full.shape), tuple(gathered.shape), tuple(logits.shape), tuple(attn.shape))
+    sys.exit(0)
diff --git a/presentation/app/requirements.txt b/presentation/app/requirements.txt
new file mode 100644
index 0000000000000000000000000000000000000000..e10422f176e08277cb6c26371f697aa42754331f
--- /dev/null
+++ b/presentation/app/requirements.txt
@@ -0,0 +1,19 @@
+# BITS Phishing Email Detector -- Hugging Face Space (Gradio SDK, ZeroGPU "zero-a10g", Python 3.12.12).
+# Pins verified live 2026-10-07 (docs/research/verify_pins.md). gradio itself is NOT listed: the Space
+# installs the version named by `sdk_version` in README.md (6.29.1). torch 2.13.0 is the highest version
+# on the ZeroGPU allow-list; huggingface_hub stays on 1.x because tokenizers 0.23.2 requires <2.0.
+torch==2.13.0
+spaces==0.51.3
+safetensors==0.8.0
+tokenizers==0.23.2
+huggingface_hub==1.33.0
+numpy==2.5.3
+
+# --- CPU-basic alternative (no ZeroGPU): swap the torch line for the CPU wheel index to skip ~3 GB of
+# --- CUDA wheels. `spaces` may stay (its decorator is a no-op off ZeroGPU) or be dropped.
+# --extra-index-url https://download.pytorch.org/whl/cpu
+# torch==2.13.0+cpu
+# safetensors==0.8.0
+# tokenizers==0.23.2
+# huggingface_hub==1.33.0
+# numpy==2.5.3
diff --git a/presentation/checkpoints/finetune/ft-12L/epochs.csv b/presentation/checkpoints/finetune/ft-12L/epochs.csv
new file mode 100644
index 0000000000000000000000000000000000000000..2935aece546dfa6db197fac818c8ab3b529c70d9
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-12L/epochs.csv
@@ -0,0 +1,9 @@
+epoch,step,is_epoch_end,n,val_loss,accuracy,precision,recall,f1,roc_auc,pr_auc,tn,fp,fn,tp,threshold,f1_optimal_threshold,f1_at_optimal_threshold,f1_unwanted,n_3way,accuracy_3way,macro_f1_3way,precision_legitimate,recall_legitimate,f1_legitimate,precision_phishing,recall_phishing,f1_phishing,precision_spam,recall_spam,f1_spam,cm3_ll,cm3_lp,cm3_ls,cm3_pl,cm3_pp,cm3_ps,cm3_sl,cm3_sp,cm3_ss,n_unwanted_3way,pvs_phish_as_phish,pvs_phish_as_spam,pvs_spam_as_phish,pvs_spam_as_spam,eval_seconds,is_best
+0,1250,0,15284,0.06674483373616884,0.9762496728605077,0.9868437231034058,0.9690895918860178,0.9778860798050564,0.9978145927327073,0.9980482662953348,6895,107,256,8026,0.5,0.21373122930526733,0.9827752348831608,0.9778860798050564,8105,0.980505860579889,0.9415535635199799,0.9942338186535966,0.9850042844901457,0.9895975321041681,0.8388746803069054,0.9647058823529412,0.8974008207934336,0.9292149292149292,0.9462647444298821,0.9376623376623376,6897,56,49,6,328,6,34,7,722,1103,334,6,9,754,25.0,1
+0,2500,0,15284,0.048256210352980757,0.9865872808165401,0.9826700131468866,0.9927553730982854,0.987686948165055,0.9988639272416802,0.9989713461053916,6857,145,60,8222,0.5,0.9050974249839783,0.9884557268056814,0.987686948165055,8105,0.9795188155459593,0.948415736923609,0.9978181818181818,0.9797200799771494,0.988686315486056,0.8975741239892183,0.9794117647058823,0.9367088607594937,0.8684516880093132,0.9777195281782438,0.9198520345252774,6860,34,108,2,333,5,13,4,746,1103,335,5,4,759,23.5,1
+0,3750,0,15284,0.06755023308291357,0.9793247840879351,0.9982486865148862,0.9635353779280367,0.9805849102973704,0.9992713642171166,0.9993947181478501,6988,14,302,7980,0.5,0.02060474082827568,0.9904222637190531,0.9805849102973704,8105,0.9898827884022209,0.964394653247903,0.9909258471572381,0.9981433876035418,0.9945215225898257,0.9801980198019802,0.8735294117647059,0.9237947122861586,0.9839786381842457,0.9659239842726082,0.9748677248677249,6989,3,10,41,297,2,23,3,737,1103,337,3,3,760,22.7,0
+0,5000,0,15284,0.035470164058502354,0.9909709500130855,0.9935757575757576,0.9897367785559044,0.9916525526252117,0.9993507650658475,0.9994415168484772,6949,53,85,8197,0.5,0.24506273865699768,0.992275196137598,0.9916525526252117,8105,0.9913633559531153,0.9754296695747446,0.9981329886543157,0.9925735504141674,0.9953455066237021,0.943502824858757,0.9823529411764705,0.962536023054755,0.9530456852791879,0.9842726081258192,0.968407479045777,6950,18,34,3,334,3,10,2,751,1103,337,3,3,760,22.6,1
+0,5009,1,15284,0.03405762961228397,0.9909709500130855,0.9919062575501328,0.9914271914996378,0.9916666666666667,0.9993699319772092,0.9994547824640465,6935,67,71,8211,0.5,0.3116289973258972,0.9919816723940436,0.9916666666666667,8105,0.989389265885256,0.9685824900641391,0.998416354736539,0.9904313053413311,0.9944078004014912,0.9245810055865922,0.9735294117647059,0.9484240687679083,0.9400749063670412,0.9868938401048493,0.9629156010230179,6935,26,41,2,331,7,9,1,753,1103,333,7,3,760,22.6,1
+1,6250,0,15284,0.035248381216093624,0.9918869405914682,0.9935866408518877,0.9914271914996378,0.9925057415689592,0.9995007118744352,0.9995737781814353,6949,53,71,8211,0.5,0.17105858027935028,0.9928829915560917,0.9925057415689592,8105,0.9916101172115978,0.9774079655816474,0.9984195402298851,0.9924307340759783,0.9954161294943418,0.9570200573065902,0.9823529411764705,0.969521044992743,0.9472361809045227,0.9882044560943644,0.9672867222578576,6949,13,40,4,334,2,7,2,754,1103,338,2,2,761,22.8,1
+1,7500,0,15284,0.067691776791314,0.9812221931431563,0.9968924798011187,0.9683651291958464,0.982421755374533,0.9994390294255459,0.999546109951094,6977,25,262,8020,0.5,0.01410032156854868,0.991664653297898,0.982421755374533,8105,0.9913633559531153,0.9745670554028227,0.9942995582157618,0.9964295915452728,0.995363435337756,0.9671641791044776,0.9529411764705882,0.96,0.9747675962815405,0.9619921363040629,0.9683377308707124,6977,9,16,13,324,3,27,2,734,1103,337,3,3,760,22.9,0
+1,8750,0,15284,0.02933201155981385,0.9916906568961005,0.9907329401853412,0.9939628109152379,0.9923452474233018,0.9995663863520969,0.9996565972351446,6925,77,50,8232,0.5,0.7635198831558228,0.9929343559393684,0.9923452474233018,8105,0.9881554595928439,0.9663859088059202,0.9984143001297391,0.9891459582976293,0.9937585192625009,0.924791086350975,0.9764705882352941,0.949928469241774,0.9283065512978986,0.9842726081258192,0.955470737913486,6926,22,54,4,332,4,7,5,751,1103,335,5,5,758,22.9,0
diff --git a/presentation/checkpoints/finetune/ft-12L/metrics.csv b/presentation/checkpoints/finetune/ft-12L/metrics.csv
new file mode 100644
index 0000000000000000000000000000000000000000..df6ccfa4415b602757f9c41fe7728f7462ecc851
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-12L/metrics.csv
@@ -0,0 +1,443 @@
+step,epoch,loss,lr,examples_per_s,max_alloc_gb,max_reserved_gb,elapsed_s,eta_s,val_loss,val_f1,val_roc_auc,val_macro_f1_3way
+20,0,0.966,1.0465116279069768e-06,112.1,6.853,6.969,5.7,2853,,,,
+40,0,0.77187,2.043189368770764e-06,125.2,6.853,6.969,10.8,2698,,,,
+60,0,0.67474,3.0398671096345517e-06,161.6,6.853,6.969,14.8,2452,,,,
+80,0,0.59153,4.036544850498338e-06,180.7,6.853,6.969,18.3,2276,,,,
+100,0,0.48763,5.033222591362126e-06,158.6,6.853,6.969,22.4,2217,,,,
+120,0,0.45492,6.029900332225914e-06,168.8,6.853,6.969,26.1,2157,,,,
+140,0,0.41796,7.026578073089702e-06,144.1,6.853,6.969,30.6,2158,,,,
+160,0,0.38791,8.023255813953488e-06,168.4,6.853,6.969,34.4,2119,,,,
+180,0,0.3301,9.019933554817277e-06,132.4,6.853,6.969,39.2,2144,,,,
+200,0,0.24755,1.0016611295681064e-05,205.8,6.853,6.969,42.3,2078,,,,
+220,0,0.27695,1.1013289036544852e-05,125.7,6.853,6.969,47.4,2112,,,,
+240,0,0.2473,1.2009966777408639e-05,163.9,6.853,6.969,51.3,2091,,,,
+260,0,0.26318,1.3006644518272425e-05,139.3,6.853,6.969,55.9,2099,,,,
+280,0,0.28014,1.4003322259136214e-05,134.3,6.853,6.969,60.7,2111,,,,
+300,0,0.22796,1.5e-05,138.1,6.853,6.969,65.3,2116,,,,
+320,0,0.22376,1.5996677740863787e-05,131.2,6.853,6.969,70.2,2128,,,,
+340,0,0.22653,1.6993355481727575e-05,129.7,6.853,6.969,75.1,2139,,,,
+360,0,0.11859,1.7990033222591364e-05,189.1,6.853,6.969,78.5,2107,,,,
+380,0,0.21683,1.898671096345515e-05,159.4,6.853,6.969,82.5,2093,,,,
+400,0,0.13274,1.9983388704318937e-05,138.5,6.853,6.969,87.2,2096,,,,
+420,0,0.11846,2.0980066445182726e-05,173.4,6.853,6.969,90.9,2076,,,,
+440,0,0.12249,2.197674418604651e-05,123.6,6.853,6.969,96.0,2090,,,,
+460,0,0.10935,2.29734219269103e-05,189.9,6.853,6.969,99.4,2065,,,,
+480,0,0.15986,2.3970099667774087e-05,139.9,6.853,6.969,104.0,2066,,,,
+500,0,0.11962,2.4966777408637872e-05,132.1,6.853,6.969,108.8,2072,,,,
+520,0,0.16062,2.596345514950166e-05,145.4,6.853,6.969,113.2,2068,,,,
+540,0,0.16635,2.696013289036545e-05,159.6,6.853,6.969,117.2,2058,,,,
+560,0,0.07012,2.7956810631229237e-05,141.2,6.853,6.969,121.8,2057,,,,
+580,0,0.15823,2.8953488372093026e-05,134.5,6.853,6.969,126.5,2059,,,,
+600,0,0.10739,2.9950166112956814e-05,148.6,6.853,6.969,130.8,2054,,,,
+620,0,0.1283,2.9942650807136788e-05,178.5,6.853,6.969,134.4,2037,,,,
+640,0,0.12727,2.9878929481733223e-05,182.0,6.853,6.969,137.9,2021,,,,
+660,0,0.07858,2.981520815632965e-05,105.8,6.853,6.969,144.0,2041,,,,
+680,0,0.1009,2.9751486830926084e-05,141.4,6.853,6.969,148.5,2039,,,,
+700,0,0.12082,2.9687765505522515e-05,146.4,6.853,6.969,152.9,2035,,,,
+720,0,0.18459,2.9624044180118947e-05,111.3,6.853,6.969,158.4,2046,,,,
+740,0,0.11558,2.956032285471538e-05,153.5,6.853,6.969,162.6,2038,,,,
+760,0,0.20052,2.949660152931181e-05,155.5,6.853,6.969,166.7,2031,,,,
+780,0,0.16744,2.9432880203908243e-05,164.9,6.853,6.969,170.6,2020,,,,
+800,0,0.10538,2.936915887850467e-05,139.3,6.853,6.969,175.2,2018,,,,
+820,0,0.09662,2.9305437553101107e-05,130.5,6.853,6.969,180.1,2020,,,,
+840,0,0.11732,2.924171622769754e-05,128.1,6.853,6.969,185.1,2022,,,,
+860,0,0.05462,2.9177994902293967e-05,177.2,6.853,6.969,188.7,2009,,,,
+880,0,0.06831,2.91142735768904e-05,139.9,6.853,6.969,193.2,2007,,,,
+900,0,0.1443,2.905055225148683e-05,142.3,6.853,6.969,197.7,2003,,,,
+920,0,0.1445,2.8986830926083263e-05,189.9,6.853,6.969,201.1,1989,,,,
+940,0,0.11019,2.8923109600679695e-05,136.7,6.853,6.969,205.8,1987,,,,
+960,0,0.05759,2.8859388275276127e-05,158.1,6.853,6.969,209.8,1980,,,,
+980,0,0.11496,2.8795666949872555e-05,155.5,6.853,6.969,214.0,1973,,,,
+1000,0,0.09381,2.873194562446899e-05,129.5,6.853,6.969,218.9,1974,,,,
+1020,0,0.08505,2.8668224299065423e-05,139.2,6.853,6.969,223.5,1972,,,,
+1040,0,0.1093,2.860450297366185e-05,132.1,6.853,6.969,228.3,1971,,,,
+1060,0,0.06057,2.8540781648258283e-05,144.8,6.853,6.969,232.8,1967,,,,
+1080,0,0.08638,2.8477060322854715e-05,123.7,6.853,6.969,237.9,1969,,,,
+1100,0,0.07907,2.841333899745115e-05,138.0,6.853,6.969,242.6,1967,,,,
+1120,0,0.08653,2.834961767204758e-05,142.6,6.853,6.969,247.1,1963,,,,
+1140,0,0.08519,2.828589634664401e-05,147.4,6.853,6.969,251.4,1958,,,,
+1160,0,0.10386,2.8222175021240443e-05,163.7,6.853,6.969,255.3,1950,,,,
+1180,0,0.07666,2.8158453695836875e-05,168.1,6.853,6.969,259.1,1941,,,,
+1200,0,0.08189,2.8094732370433307e-05,150.2,6.853,6.969,263.4,1935,,,,
+1220,0,0.05436,2.803101104502974e-05,167.8,6.853,6.969,267.2,1927,,,,
+1240,0,0.08977,2.7967289719626167e-05,121.5,6.853,6.969,272.5,1929,,,,
+1250,0,0.0719,2.7935429056924385e-05,185.7,6.853,6.969,274.2,1923,0.06674,0.97789,0.99781,0.94155
+1260,0,0.02679,2.79035683942226e-05,186.8,6.853,6.969,304.3,1918,,,,
+1280,0,0.09069,2.7839847068819034e-05,121.9,6.853,6.969,309.6,1919,,,,
+1300,0,0.1034,2.7776125743415463e-05,143.3,6.853,6.969,314.0,1915,,,,
+1320,0,0.08424,2.7712404418011895e-05,131.9,6.853,6.969,318.9,1914,,,,
+1340,0,0.06766,2.7648683092608327e-05,153.3,6.853,6.969,323.1,1908,,,,
+1360,0,0.08319,2.758496176720476e-05,130.6,6.853,6.969,328.0,1907,,,,
+1380,0,0.04261,2.752124044180119e-05,133.5,6.853,6.969,332.7,1905,,,,
+1400,0,0.07709,2.7457519116397622e-05,126.8,6.853,6.969,337.8,1905,,,,
+1420,0,0.08428,2.739379779099405e-05,137.4,6.853,6.969,342.5,1902,,,,
+1440,0,0.05155,2.7330076465590486e-05,130.7,6.853,6.969,347.3,1900,,,,
+1460,0,0.09329,2.7266355140186918e-05,121.5,6.853,6.969,352.6,1900,,,,
+1480,0,0.06021,2.720263381478335e-05,131.4,6.853,6.969,357.5,1898,,,,
+1500,0,0.05044,2.713891248937978e-05,146.0,6.853,6.969,361.9,1894,,,,
+1520,0,0.09607,2.707519116397621e-05,144.8,6.853,6.969,366.3,1889,,,,
+1540,0,0.05835,2.7011469838572646e-05,145.3,6.853,6.969,370.7,1884,,,,
+1560,0,0.06545,2.6947748513169074e-05,159.0,6.853,6.969,374.7,1878,,,,
+1580,0,0.04257,2.6884027187765506e-05,124.0,6.853,6.969,379.9,1877,,,,
+1600,0,0.0787,2.6820305862361938e-05,145.3,6.853,6.969,384.3,1872,,,,
+1620,0,0.06848,2.675658453695837e-05,124.0,6.853,6.969,389.4,1872,,,,
+1640,0,0.06393,2.6692863211554802e-05,137.8,6.853,6.969,394.1,1868,,,,
+1660,0,0.04771,2.6629141886151234e-05,148.9,6.853,6.969,398.4,1863,,,,
+1680,0,0.07965,2.6565420560747662e-05,122.1,6.853,6.969,403.6,1862,,,,
+1700,0,0.06073,2.6501699235344094e-05,115.0,6.853,6.969,409.2,1863,,,,
+1720,0,0.07176,2.643797790994053e-05,140.7,6.853,6.969,413.7,1859,,,,
+1740,0,0.05617,2.6374256584536958e-05,136.0,6.853,6.969,418.4,1856,,,,
+1760,0,0.0521,2.631053525913339e-05,130.1,6.853,6.969,423.4,1853,,,,
+1780,0,0.03388,2.6246813933729822e-05,149.4,6.853,6.969,427.7,1848,,,,
+1800,0,0.07185,2.6183092608326254e-05,139.4,6.853,6.969,432.2,1844,,,,
+1820,0,0.04105,2.6119371282922686e-05,142.5,6.853,6.969,436.7,1839,,,,
+1840,0,0.03656,2.6055649957519118e-05,186.2,6.853,6.969,440.2,1830,,,,
+1860,0,0.04553,2.599192863211555e-05,153.0,6.853,6.969,444.4,1824,,,,
+1880,0,0.05058,2.5928207306711978e-05,112.8,6.853,6.969,450.0,1825,,,,
+1900,0,0.07056,2.5864485981308413e-05,148.7,6.853,6.969,454.3,1820,,,,
+1920,0,0.04508,2.5800764655904845e-05,132.1,6.853,6.969,459.2,1817,,,,
+1940,0,0.05152,2.5737043330501274e-05,157.9,6.853,6.969,463.2,1811,,,,
+1960,0,0.01702,2.5673322005097706e-05,172.9,6.853,6.969,466.9,1803,,,,
+1980,0,0.09522,2.560960067969414e-05,126.0,6.853,6.969,472.0,1801,,,,
+2000,0,0.0717,2.554587935429057e-05,134.6,6.853,6.969,476.8,1797,,,,
+2020,0,0.04202,2.5482158028887e-05,125.9,6.853,6.969,481.8,1795,,,,
+2040,0,0.07226,2.5418436703483433e-05,135.8,6.853,6.969,486.6,1792,,,,
+2060,0,0.03688,2.5354715378079862e-05,121.8,6.853,6.969,491.8,1790,,,,
+2080,0,0.05288,2.5290994052676297e-05,119.0,6.853,6.969,497.2,1789,,,,
+2100,0,0.07093,2.522727272727273e-05,127.4,6.853,6.969,502.2,1786,,,,
+2120,0,0.0557,2.5163551401869158e-05,122.5,6.853,6.969,507.4,1785,,,,
+2140,0,0.07049,2.509983007646559e-05,106.4,6.853,6.969,513.5,1786,,,,
+2160,0,0.0557,2.5036108751062025e-05,134.6,6.853,6.969,518.2,1782,,,,
+2180,0,0.04015,2.4972387425658453e-05,145.6,6.853,6.969,522.6,1777,,,,
+2200,0,0.05014,2.4908666100254885e-05,148.5,6.853,6.969,526.9,1772,,,,
+2220,0,0.05217,2.4844944774851317e-05,116.6,6.853,6.969,532.4,1770,,,,
+2240,0,0.04526,2.478122344944775e-05,153.8,6.853,6.969,536.6,1765,,,,
+2260,0,0.05863,2.471750212404418e-05,125.0,6.853,6.969,541.7,1762,,,,
+2280,0,0.08123,2.4653780798640613e-05,120.8,6.853,6.969,547.0,1760,,,,
+2300,0,0.04107,2.4590059473237045e-05,134.2,6.853,6.969,551.7,1756,,,,
+2320,0,0.04968,2.4526338147833473e-05,140.7,6.853,6.969,556.3,1752,,,,
+2340,0,0.03855,2.446261682242991e-05,162.3,6.853,6.969,560.2,1745,,,,
+2360,0,0.06711,2.439889549702634e-05,123.3,6.853,6.969,565.4,1743,,,,
+2380,0,0.02658,2.433517417162277e-05,141.3,6.853,6.969,570.0,1738,,,,
+2400,0,0.03456,2.42714528462192e-05,152.8,6.853,6.969,574.2,1732,,,,
+2420,0,0.06754,2.4207731520815636e-05,164.0,6.853,6.969,578.1,1726,,,,
+2440,0,0.01897,2.4144010195412065e-05,161.8,6.853,6.969,582.0,1719,,,,
+2460,0,0.10267,2.4080288870008497e-05,124.7,6.853,6.969,587.1,1717,,,,
+2480,0,0.06007,2.401656754460493e-05,124.7,6.853,6.969,592.3,1714,,,,
+2500,0,0.02989,2.3952846219201357e-05,131.0,6.853,6.969,597.2,1710,0.04826,0.98769,0.99886,0.94842
+2520,0,0.06986,2.3889124893797792e-05,144.8,6.853,6.969,628.9,1705,,,,
+2540,0,0.04839,2.3825403568394224e-05,136.1,6.853,6.969,633.6,1701,,,,
+2560,0,0.07039,2.3761682242990653e-05,139.1,6.853,6.969,638.2,1697,,,,
+2580,0,0.03076,2.3697960917587085e-05,175.7,6.853,6.969,641.9,1690,,,,
+2600,0,0.04476,2.363423959218352e-05,142.3,6.853,6.969,646.4,1685,,,,
+2620,0,0.04998,2.3570518266779952e-05,133.2,6.853,6.969,651.2,1681,,,,
+2640,0,0.04793,2.350679694137638e-05,163.4,6.853,6.969,655.1,1675,,,,
+2660,0,0.04456,2.3443075615972812e-05,186.8,6.853,6.969,658.5,1667,,,,
+2680,0,0.03933,2.3379354290569244e-05,153.0,6.853,6.969,662.7,1662,,,,
+2700,0,0.05857,2.3315632965165676e-05,135.0,6.853,6.969,667.4,1658,,,,
+2720,0,0.06571,2.3251911639762108e-05,161.1,6.853,6.969,671.4,1652,,,,
+2740,0,0.05786,2.318819031435854e-05,135.7,6.853,6.969,676.1,1648,,,,
+2760,0,0.04993,2.312446898895497e-05,144.8,6.853,6.969,680.6,1643,,,,
+2780,0,0.05783,2.3060747663551404e-05,152.6,6.853,6.969,684.8,1638,,,,
+2800,0,0.05139,2.2997026338147836e-05,126.8,6.853,6.969,689.8,1634,,,,
+2820,0,0.0351,2.2933305012744264e-05,177.3,6.853,6.969,693.4,1628,,,,
+2840,0,0.04243,2.2869583687340696e-05,124.0,6.853,6.969,698.6,1625,,,,
+2860,0,0.06953,2.2805862361937128e-05,129.4,6.853,6.969,703.5,1621,,,,
+2880,0,0.02242,2.274214103653356e-05,157.7,6.853,6.969,707.6,1616,,,,
+2900,0,0.04502,2.2678419711129992e-05,164.6,6.853,6.969,711.5,1609,,,,
+2920,0,0.05603,2.2614698385726424e-05,164.0,6.853,6.969,715.4,1603,,,,
+2940,0,0.07678,2.2550977060322852e-05,127.7,6.853,6.969,720.4,1600,,,,
+2960,0,0.04243,2.2487255734919288e-05,156.6,6.853,6.969,724.5,1595,,,,
+2980,0,0.01146,2.242353440951572e-05,190.6,6.853,6.969,727.8,1587,,,,
+3000,0,0.03996,2.235981308411215e-05,137.5,6.853,6.969,732.5,1583,,,,
+3020,0,0.02901,2.229609175870858e-05,178.3,6.853,6.969,736.1,1576,,,,
+3040,0,0.05288,2.2232370433305012e-05,155.9,6.853,6.969,740.2,1571,,,,
+3060,0,0.0255,2.2168649107901447e-05,163.1,6.853,6.969,744.1,1565,,,,
+3080,0,0.06377,2.2104927782497876e-05,134.8,6.853,6.969,748.8,1561,,,,
+3100,0,0.03137,2.2041206457094308e-05,132.4,6.853,6.969,753.7,1557,,,,
+3120,0,0.03418,2.197748513169074e-05,142.5,6.853,6.969,758.2,1553,,,,
+3140,0,0.05617,2.191376380628717e-05,142.9,6.853,6.969,762.7,1548,,,,
+3160,0,0.0335,2.1850042480883604e-05,171.2,6.853,6.969,766.4,1542,,,,
+3180,0,0.03702,2.1786321155480035e-05,146.6,6.853,6.969,770.8,1537,,,,
+3200,0,0.08364,2.1722599830076464e-05,144.3,6.853,6.969,775.2,1533,,,,
+3220,0,0.04111,2.16588785046729e-05,153.9,6.853,6.969,779.3,1528,,,,
+3240,0,0.02646,2.159515717926933e-05,142.1,6.853,6.969,783.9,1523,,,,
+3260,0,0.05342,2.153143585386576e-05,145.7,6.853,6.969,788.2,1518,,,,
+3280,0,0.02546,2.146771452846219e-05,138.3,6.853,6.969,792.9,1514,,,,
+3300,0,0.04305,2.1403993203058624e-05,180.1,6.853,6.969,796.4,1508,,,,
+3320,0,0.02875,2.1340271877655055e-05,154.0,6.853,6.969,800.6,1503,,,,
+3340,0,0.03978,2.1276550552251487e-05,211.6,6.853,6.969,803.6,1495,,,,
+3360,0,0.02442,2.121282922684792e-05,139.4,6.853,6.969,808.2,1491,,,,
+3380,0,0.0461,2.1149107901444348e-05,141.4,6.853,6.969,812.7,1487,,,,
+3400,0,0.07481,2.1085386576040783e-05,168.2,6.853,6.969,816.5,1481,,,,
+3420,0,0.07186,2.1021665250637215e-05,153.2,6.853,6.969,820.7,1476,,,,
+3440,0,0.05047,2.0957943925233647e-05,143.3,6.853,6.969,825.2,1471,,,,
+3460,0,0.01579,2.0894222599830075e-05,152.8,6.853,6.969,829.4,1466,,,,
+3480,0,0.06416,2.0830501274426507e-05,121.1,6.853,6.969,834.6,1463,,,,
+3500,0,0.03135,2.0766779949022943e-05,177.9,6.853,6.969,838.2,1457,,,,
+3520,0,0.04428,2.070305862361937e-05,199.1,6.853,6.969,841.5,1450,,,,
+3540,0,0.05385,2.0639337298215803e-05,161.0,6.853,6.969,845.4,1445,,,,
+3560,0,0.04246,2.0575615972812235e-05,167.0,6.853,6.969,849.3,1439,,,,
+3580,0,0.03012,2.0511894647408667e-05,156.7,6.853,6.969,853.3,1434,,,,
+3600,0,0.03364,2.04481733220051e-05,140.6,6.853,6.969,857.9,1430,,,,
+3620,0,0.04835,2.038445199660153e-05,145.9,6.853,6.969,862.3,1425,,,,
+3640,0,0.03662,2.032073067119796e-05,201.6,6.853,6.969,865.5,1419,,,,
+3660,0,0.04867,2.025700934579439e-05,158.5,6.853,6.969,869.5,1414,,,,
+3680,0,0.07107,2.0193288020390827e-05,145.1,6.853,6.969,873.9,1409,,,,
+3700,0,0.02278,2.0129566694987255e-05,179.4,6.853,6.969,877.5,1403,,,,
+3720,0,0.02047,2.0065845369583687e-05,145.2,6.853,6.969,881.9,1399,,,,
+3740,0,0.08711,2.000212404418012e-05,142.3,6.853,6.969,886.4,1394,,,,
+3750,0,0.05831,1.9970263381478337e-05,159.1,6.853,6.969,888.4,1392,0.06755,0.98058,0.99927,0.96439
+3760,0,0.04379,1.993840271877655e-05,153.7,6.853,6.969,915.6,1389,,,,
+3780,0,0.03137,1.9874681393372983e-05,162.5,6.853,6.969,919.5,1384,,,,
+3800,0,0.04044,1.9810960067969415e-05,154.0,6.853,6.969,923.7,1379,,,,
+3820,0,0.04398,1.9747238742565847e-05,147.7,6.853,6.969,928.0,1375,,,,
+3840,0,0.03757,1.9683517417162275e-05,165.0,6.853,6.969,931.9,1369,,,,
+3860,0,0.01506,1.961979609175871e-05,143.4,6.853,6.969,936.3,1365,,,,
+3880,0,0.03935,1.9556074766355142e-05,159.2,6.853,6.969,940.3,1360,,,,
+3900,0,0.02473,1.949235344095157e-05,150.7,6.853,6.969,944.6,1355,,,,
+3920,0,0.05283,1.9428632115548003e-05,150.2,6.853,6.969,948.9,1350,,,,
+3940,0,0.05914,1.9364910790144438e-05,129.8,6.853,6.969,953.8,1347,,,,
+3960,0,0.02901,1.9301189464740867e-05,161.1,6.853,6.969,957.8,1342,,,,
+3980,0,0.09993,1.92374681393373e-05,176.5,6.853,6.969,961.4,1336,,,,
+4000,0,0.01099,1.917374681393373e-05,144.8,6.853,6.969,965.8,1331,,,,
+4020,0,0.04233,1.9110025488530162e-05,149.7,6.853,6.969,970.1,1327,,,,
+4040,0,0.04254,1.9046304163126594e-05,135.5,6.853,6.969,974.8,1323,,,,
+4060,0,0.04043,1.8982582837723026e-05,158.9,6.853,6.969,978.8,1318,,,,
+4080,0,0.06333,1.8918861512319455e-05,122.8,6.853,6.969,984.0,1315,,,,
+4100,0,0.01552,1.8855140186915887e-05,152.0,6.853,6.969,988.3,1310,,,,
+4120,0,0.02354,1.8791418861512322e-05,156.7,6.853,6.969,992.3,1305,,,,
+4140,0,0.06064,1.872769753610875e-05,137.5,6.853,6.969,997.0,1301,,,,
+4160,0,0.06429,1.8663976210705182e-05,147.5,6.853,6.969,1001.3,1296,,,,
+4180,0,0.05014,1.8600254885301614e-05,136.7,6.853,6.969,1006.0,1292,,,,
+4200,0,0.03012,1.853653355989805e-05,155.3,6.853,6.969,1010.1,1287,,,,
+4220,0,0.01509,1.8472812234494478e-05,199.9,6.853,6.969,1013.3,1281,,,,
+4240,0,0.0393,1.840909090909091e-05,187.8,6.853,6.969,1016.7,1275,,,,
+4260,0,0.04395,1.8345369583687342e-05,124.4,6.853,6.969,1021.9,1272,,,,
+4280,0,0.05886,1.828164825828377e-05,161.2,6.853,6.969,1025.9,1267,,,,
+4300,0,0.02988,1.8217926932880206e-05,126.5,6.853,6.969,1030.9,1263,,,,
+4320,0,0.04604,1.8154205607476638e-05,169.5,6.853,6.969,1034.7,1258,,,,
+4340,0,0.01279,1.8090484282073066e-05,132.3,6.853,6.969,1039.5,1254,,,,
+4360,0,0.0401,1.8026762956669498e-05,129.1,6.853,6.969,1044.5,1251,,,,
+4380,0,0.00727,1.7963041631265933e-05,136.1,6.853,6.969,1049.2,1246,,,,
+4400,0,0.0722,1.7899320305862362e-05,130.7,6.853,6.969,1054.1,1243,,,,
+4420,0,0.03825,1.7835598980458794e-05,144.5,6.853,6.969,1058.5,1238,,,,
+4440,0,0.03648,1.7771877655055226e-05,171.2,6.853,6.969,1062.3,1233,,,,
+4460,0,0.02956,1.7708156329651654e-05,148.4,6.853,6.969,1066.6,1228,,,,
+4480,0,0.0485,1.764443500424809e-05,136.0,6.853,6.969,1071.3,1224,,,,
+4500,0,0.01765,1.758071367884452e-05,154.4,6.853,6.969,1075.4,1220,,,,
+4520,0,0.021,1.751699235344095e-05,142.6,6.853,6.969,1079.9,1215,,,,
+4540,0,0.02802,1.7453271028037382e-05,163.3,6.853,6.969,1083.8,1210,,,,
+4560,0,0.03997,1.7389549702633817e-05,127.3,6.853,6.969,1088.9,1207,,,,
+4580,0,0.0461,1.732582837723025e-05,132.4,6.853,6.969,1093.7,1203,,,,
+4600,0,0.06121,1.7262107051826678e-05,131.2,6.853,6.969,1098.6,1199,,,,
+4620,0,0.02779,1.719838572642311e-05,140.6,6.853,6.969,1103.1,1194,,,,
+4640,0,0.03123,1.713466440101954e-05,205.1,6.853,6.969,1106.2,1189,,,,
+4660,0,0.07945,1.7070943075615973e-05,134.7,6.853,6.969,1111.0,1184,,,,
+4680,0,0.02676,1.7007221750212405e-05,158.9,6.853,6.969,1115.0,1180,,,,
+4700,0,0.01955,1.6943500424808837e-05,167.0,6.853,6.969,1118.9,1175,,,,
+4720,0,0.04432,1.6879779099405266e-05,132.0,6.853,6.969,1123.7,1171,,,,
+4740,0,0.02774,1.68160577740017e-05,148.1,6.853,6.969,1128.0,1166,,,,
+4760,0,0.06885,1.6752336448598133e-05,148.2,6.853,6.969,1132.3,1162,,,,
+4780,0,0.03827,1.668861512319456e-05,177.2,6.853,6.969,1136.0,1156,,,,
+4800,0,0.01952,1.6624893797790993e-05,152.0,6.853,6.969,1140.2,1152,,,,
+4820,0,0.03209,1.6561172472387425e-05,148.9,6.853,6.969,1144.5,1147,,,,
+4840,0,0.02674,1.6497451146983857e-05,121.6,6.853,6.969,1149.7,1144,,,,
+4860,0,0.02004,1.643372982158029e-05,172.8,6.853,6.969,1153.4,1138,,,,
+4880,0,0.01364,1.637000849617672e-05,154.1,6.853,6.969,1157.6,1134,,,,
+4900,0,0.03521,1.630628717077315e-05,130.2,6.853,6.969,1162.5,1130,,,,
+4920,0,0.03462,1.6242565845369585e-05,139.2,6.853,6.969,1167.1,1126,,,,
+4940,0,0.0193,1.6178844519966017e-05,151.1,6.853,6.969,1171.3,1121,,,,
+4960,0,0.02533,1.611512319456245e-05,131.2,6.853,6.969,1176.2,1117,,,,
+4980,0,0.05521,1.6051401869158877e-05,136.3,6.853,6.969,1180.9,1113,,,,
+5000,0,0.02881,1.5987680543755312e-05,150.0,6.853,6.969,1185.2,1108,0.03547,0.99165,0.99935,0.97543
+5009,0,0.10062,1.5959005947323706e-05,195.2,6.853,6.969,1212.9,1106,0.03406,0.99167,0.99937,0.96858
+5020,1,0.02308,1.5923959218351744e-05,148.1,6.853,6.969,1240.9,1103,,,,
+5040,1,0.04093,1.5860237892948173e-05,159.2,6.853,6.969,1244.9,1099,,,,
+5060,1,0.01192,1.5796516567544605e-05,153.6,6.853,6.969,1249.1,1094,,,,
+5080,1,0.04088,1.5732795242141037e-05,134.4,6.853,6.969,1253.9,1090,,,,
+5100,1,0.02177,1.566907391673747e-05,181.5,6.853,6.969,1257.4,1085,,,,
+5120,1,0.02735,1.56053525913339e-05,158.6,6.853,6.969,1261.4,1080,,,,
+5140,1,0.01635,1.5541631265930333e-05,141.8,6.853,6.969,1265.9,1075,,,,
+5160,1,0.01509,1.547790994052676e-05,171.6,6.853,6.969,1269.7,1070,,,,
+5180,1,0.02187,1.5414188615123196e-05,226.6,6.853,6.969,1272.5,1065,,,,
+5200,1,0.02184,1.5350467289719628e-05,142.8,6.853,6.969,1277.0,1060,,,,
+5220,1,0.01002,1.5286745964316057e-05,170.3,6.853,6.969,1280.7,1055,,,,
+5240,1,0.02312,1.5223024638912489e-05,130.4,6.853,6.969,1285.6,1051,,,,
+5260,1,0.04426,1.5159303313508922e-05,156.8,6.853,6.969,1289.7,1047,,,,
+5280,1,0.01664,1.509558198810535e-05,151.8,6.853,6.969,1293.9,1042,,,,
+5300,1,0.01596,1.5031860662701784e-05,129.2,6.853,6.969,1298.9,1038,,,,
+5320,1,0.0146,1.4968139337298216e-05,121.2,6.853,6.969,1304.2,1034,,,,
+5340,1,0.02616,1.4904418011894648e-05,149.0,6.853,6.969,1308.5,1030,,,,
+5360,1,0.02075,1.4840696686491078e-05,150.1,6.853,6.969,1312.7,1025,,,,
+5380,1,0.02048,1.4776975361087512e-05,131.4,6.853,6.969,1317.6,1021,,,,
+5400,1,0.00972,1.4713254035683942e-05,131.1,6.853,6.969,1322.5,1017,,,,
+5420,1,0.02065,1.4649532710280374e-05,142.1,6.853,6.969,1327.0,1013,,,,
+5440,1,0.03307,1.4585811384876806e-05,141.4,6.853,6.969,1331.5,1009,,,,
+5460,1,0.05114,1.4522090059473236e-05,141.6,6.853,6.969,1336.0,1005,,,,
+5480,1,0.01417,1.445836873406967e-05,140.8,6.853,6.969,1340.6,1000,,,,
+5500,1,0.01634,1.43946474086661e-05,153.4,6.853,6.969,1344.7,996,,,,
+5520,1,0.01185,1.4330926083262532e-05,179.5,6.853,6.969,1348.3,991,,,,
+5540,1,0.01076,1.4267204757858964e-05,127.9,6.853,6.969,1353.3,987,,,,
+5560,1,0.02773,1.4203483432455396e-05,167.0,6.853,6.969,1357.1,982,,,,
+5580,1,0.03328,1.4139762107051826e-05,264.8,6.853,6.969,1359.6,976,,,,
+5600,1,0.02733,1.407604078164826e-05,174.8,6.853,6.969,1363.2,971,,,,
+5620,1,0.0247,1.401231945624469e-05,148.6,6.853,6.969,1367.5,966,,,,
+5640,1,0.01191,1.3948598130841122e-05,155.9,6.853,6.969,1371.6,962,,,,
+5660,1,0.02901,1.3884876805437554e-05,136.4,6.853,6.969,1376.3,958,,,,
+5680,1,0.01421,1.3821155480033984e-05,154.1,6.853,6.969,1380.5,953,,,,
+5700,1,0.04741,1.3757434154630418e-05,144.2,6.853,6.969,1384.9,949,,,,
+5720,1,0.01512,1.3693712829226848e-05,165.0,6.853,6.969,1388.8,944,,,,
+5740,1,0.0318,1.362999150382328e-05,186.3,6.853,6.969,1392.2,939,,,,
+5760,1,0.00622,1.3566270178419712e-05,135.8,6.853,6.969,1396.9,935,,,,
+5780,1,0.04705,1.3502548853016144e-05,136.4,6.853,6.969,1401.6,930,,,,
+5800,1,0.03234,1.3438827527612575e-05,165.0,6.853,6.969,1405.5,926,,,,
+5820,1,0.01046,1.3375106202209006e-05,126.6,6.853,6.969,1410.6,922,,,,
+5840,1,0.01974,1.3311384876805438e-05,147.8,6.853,6.969,1414.9,917,,,,
+5860,1,0.02331,1.324766355140187e-05,148.2,6.853,6.969,1419.2,913,,,,
+5880,1,0.0217,1.3183942225998301e-05,131.5,6.853,6.969,1424.1,909,,,,
+5900,1,0.02454,1.3120220900594732e-05,147.9,6.853,6.969,1428.4,904,,,,
+5920,1,0.01337,1.3056499575191165e-05,138.7,6.853,6.969,1433.0,900,,,,
+5940,1,0.01525,1.2992778249787595e-05,138.2,6.853,6.969,1437.6,896,,,,
+5960,1,0.029,1.2929056924384027e-05,135.3,6.853,6.969,1442.4,892,,,,
+5980,1,0.03691,1.286533559898046e-05,138.7,6.853,6.969,1447.0,887,,,,
+6000,1,0.03168,1.2801614273576891e-05,129.6,6.853,6.969,1451.9,883,,,,
+6020,1,0.0289,1.2737892948173323e-05,159.5,6.853,6.969,1455.9,879,,,,
+6040,1,0.00664,1.2674171622769753e-05,159.4,6.853,6.969,1460.0,874,,,,
+6060,1,0.0135,1.2610450297366185e-05,132.8,6.853,6.969,1464.8,870,,,,
+6080,1,0.01978,1.2546728971962617e-05,168.4,6.853,6.969,1468.6,865,,,,
+6100,1,0.02989,1.2483007646559049e-05,137.9,6.853,6.969,1473.2,861,,,,
+6120,1,0.00436,1.241928632115548e-05,151.7,6.853,6.969,1477.4,856,,,,
+6140,1,0.02784,1.2355564995751913e-05,195.1,6.853,6.969,1480.7,851,,,,
+6160,1,0.00141,1.2291843670348343e-05,138.1,6.853,6.969,1485.4,847,,,,
+6180,1,0.01232,1.2228122344944777e-05,146.5,6.853,6.969,1489.7,843,,,,
+6200,1,0.02618,1.2164401019541207e-05,145.8,6.853,6.969,1494.1,838,,,,
+6220,1,0.02083,1.2100679694137637e-05,164.7,6.853,6.969,1498.0,834,,,,
+6240,1,0.01728,1.203695836873407e-05,168.1,6.853,6.969,1501.8,829,,,,
+6250,1,0.02432,1.2005097706032285e-05,101.3,6.853,6.969,1505.0,827,0.03525,0.99251,0.9995,0.97741
+6260,1,0.03302,1.1973237043330501e-05,152.3,6.853,6.969,1533.5,825,,,,
+6280,1,0.01317,1.1909515717926933e-05,219.7,6.853,6.969,1536.5,820,,,,
+6300,1,0.03827,1.1845794392523365e-05,141.1,6.853,6.969,1541.0,815,,,,
+6320,1,0.0056,1.1782073067119797e-05,131.7,6.853,6.969,1545.8,811,,,,
+6340,1,0.00494,1.1718351741716227e-05,164.1,6.853,6.969,1549.7,807,,,,
+6360,1,0.03622,1.165463041631266e-05,239.3,6.853,6.969,1552.4,801,,,,
+6380,1,0.00338,1.159090909090909e-05,147.5,6.853,6.969,1556.8,797,,,,
+6400,1,0.02158,1.1527187765505523e-05,186.0,6.853,6.969,1560.2,792,,,,
+6420,1,0.0339,1.1463466440101955e-05,140.5,6.853,6.969,1564.8,788,,,,
+6440,1,0.04125,1.1399745114698385e-05,133.7,6.853,6.969,1569.5,784,,,,
+6460,1,0.01043,1.1336023789294818e-05,154.7,6.853,6.969,1573.7,779,,,,
+6480,1,0.00861,1.1272302463891249e-05,149.7,6.853,6.969,1578.0,775,,,,
+6500,1,0.00628,1.120858113848768e-05,182.3,6.853,6.969,1581.5,770,,,,
+6520,1,0.02944,1.1144859813084113e-05,160.5,6.853,6.969,1585.5,765,,,,
+6540,1,0.03702,1.1081138487680544e-05,154.2,6.853,6.969,1589.6,761,,,,
+6560,1,0.01429,1.1017417162276975e-05,173.4,6.853,6.969,1593.3,756,,,,
+6580,1,0.02205,1.0953695836873408e-05,144.7,6.853,6.969,1597.7,752,,,,
+6600,1,0.01856,1.0889974511469838e-05,149.6,6.853,6.969,1602.0,747,,,,
+6620,1,0.01039,1.082625318606627e-05,123.4,6.853,6.969,1607.2,743,,,,
+6640,1,0.00262,1.0762531860662702e-05,120.0,6.853,6.969,1612.5,739,,,,
+6660,1,0.02606,1.0698810535259133e-05,195.2,6.853,6.969,1615.8,734,,,,
+6680,1,0.01871,1.0635089209855566e-05,157.6,6.853,6.969,1619.9,730,,,,
+6700,1,0.02234,1.0571367884451996e-05,143.8,6.853,6.969,1624.3,726,,,,
+6720,1,0.01779,1.0507646559048428e-05,133.4,6.853,6.969,1629.1,721,,,,
+6740,1,0.00399,1.044392523364486e-05,142.0,6.853,6.969,1633.6,717,,,,
+6760,1,0.01463,1.0380203908241292e-05,131.5,6.853,6.969,1638.5,713,,,,
+6780,1,0.01516,1.0316482582837724e-05,134.3,6.853,6.969,1643.2,709,,,,
+6800,1,0.02252,1.0252761257434154e-05,162.2,6.853,6.969,1647.2,704,,,,
+6820,1,0.02035,1.0189039932030586e-05,124.6,6.853,6.969,1652.3,700,,,,
+6840,1,0.01068,1.0125318606627018e-05,178.7,6.853,6.969,1655.9,695,,,,
+6860,1,0.02681,1.006159728122345e-05,146.8,6.853,6.969,1660.3,691,,,,
+6880,1,0.01091,9.99787595581988e-06,172.7,6.853,6.969,1664.0,686,,,,
+6900,1,0.00305,9.934154630416314e-06,173.9,6.853,6.969,1667.7,682,,,,
+6920,1,0.02587,9.870433305012744e-06,134.9,6.853,6.969,1672.4,677,,,,
+6940,1,0.02036,9.806711979609176e-06,127.2,6.853,6.969,1677.4,673,,,,
+6960,1,0.01864,9.742990654205608e-06,177.5,6.853,6.969,1681.0,669,,,,
+6980,1,0.01156,9.67926932880204e-06,175.9,6.853,6.969,1684.7,664,,,,
+7000,1,0.01507,9.615548003398472e-06,137.7,6.853,6.969,1689.3,660,,,,
+7020,1,0.01741,9.551826677994902e-06,176.0,6.853,6.969,1693.0,655,,,,
+7040,1,0.02575,9.488105352591334e-06,163.2,6.853,6.969,1696.9,650,,,,
+7060,1,0.01903,9.424384027187766e-06,134.7,6.853,6.969,1701.6,646,,,,
+7080,1,0.0386,9.360662701784198e-06,154.0,6.853,6.969,1705.8,642,,,,
+7100,1,0.00136,9.296941376380628e-06,144.0,6.853,6.969,1710.2,637,,,,
+7120,1,0.00427,9.233220050977061e-06,154.3,6.853,6.969,1714.4,633,,,,
+7140,1,0.01825,9.169498725573492e-06,141.5,6.853,6.969,1718.9,629,,,,
+7160,1,0.01798,9.105777400169925e-06,120.3,6.853,6.969,1724.2,625,,,,
+7180,1,0.01884,9.042056074766356e-06,174.3,6.853,6.969,1727.9,620,,,,
+7200,1,0.02123,8.978334749362786e-06,119.8,6.853,6.969,1733.2,616,,,,
+7220,1,0.00347,8.91461342395922e-06,160.2,6.853,6.969,1737.2,612,,,,
+7240,1,0.0044,8.85089209855565e-06,214.3,6.853,6.969,1740.2,607,,,,
+7260,1,0.02341,8.787170773152081e-06,119.7,6.853,6.969,1745.6,603,,,,
+7280,1,0.02172,8.723449447748513e-06,131.6,6.853,6.969,1750.4,598,,,,
+7300,1,0.00203,8.659728122344945e-06,124.4,6.853,6.969,1755.6,594,,,,
+7320,1,0.01416,8.596006796941376e-06,181.0,6.853,6.969,1759.1,590,,,,
+7340,1,0.00626,8.532285471537809e-06,173.6,6.853,6.969,1762.8,585,,,,
+7360,1,0.01079,8.46856414613424e-06,154.8,6.853,6.969,1766.9,581,,,,
+7380,1,0.01489,8.404842820730673e-06,175.8,6.853,6.969,1770.6,576,,,,
+7400,1,0.00776,8.341121495327103e-06,144.7,6.853,6.969,1775.0,572,,,,
+7420,1,0.00938,8.277400169923533e-06,147.1,6.853,6.969,1779.3,567,,,,
+7440,1,0.02455,8.213678844519967e-06,137.9,6.853,6.969,1784.0,563,,,,
+7460,1,0.04173,8.149957519116397e-06,142.3,6.853,6.969,1788.5,559,,,,
+7480,1,0.03365,8.086236193712829e-06,184.3,6.853,6.969,1792.0,554,,,,
+7500,1,0.01488,8.022514868309261e-06,157.9,6.853,6.969,1796.0,550,0.06769,0.98242,0.99944,0.97457
+7520,1,0.01368,7.958793542905693e-06,128.9,6.853,6.969,1826.1,545,,,,
+7540,1,0.01705,7.895072217502125e-06,144.7,6.853,6.969,1830.5,541,,,,
+7560,1,0.02758,7.831350892098557e-06,155.2,6.853,6.969,1834.6,537,,,,
+7580,1,0.02697,7.767629566694987e-06,127.1,6.853,6.969,1839.6,532,,,,
+7600,1,0.01976,7.703908241291419e-06,161.3,6.853,6.969,1843.6,528,,,,
+7620,1,0.00205,7.64018691588785e-06,186.9,6.853,6.969,1847.0,523,,,,
+7640,1,0.00116,7.576465590484282e-06,164.4,6.853,6.969,1850.9,519,,,,
+7660,1,0.01931,7.512744265080715e-06,134.7,6.853,6.969,1855.7,515,,,,
+7680,1,0.00401,7.449022939677146e-06,118.4,6.853,6.969,1861.1,510,,,,
+7700,1,0.00134,7.385301614273577e-06,137.7,6.853,6.969,1865.7,506,,,,
+7720,1,0.01291,7.321580288870009e-06,144.6,6.853,6.969,1870.2,502,,,,
+7740,1,0.02277,7.25785896346644e-06,143.7,6.853,6.969,1874.6,497,,,,
+7760,1,0.02703,7.194137638062872e-06,184.7,6.853,6.969,1878.1,493,,,,
+7780,1,0.02033,7.130416312659304e-06,151.6,6.853,6.969,1882.3,488,,,,
+7800,1,0.02029,7.0666949872557355e-06,146.8,6.853,6.969,1886.7,484,,,,
+7820,1,0.02051,7.0029736618521666e-06,146.4,6.853,6.969,1891.0,480,,,,
+7840,1,0.00665,6.9392523364485985e-06,160.3,6.853,6.969,1895.0,475,,,,
+7860,1,0.0008,6.87553101104503e-06,151.1,6.853,6.969,1899.3,471,,,,
+7880,1,0.02083,6.811809685641462e-06,133.3,6.853,6.969,1904.1,467,,,,
+7900,1,0.02276,6.7480883602378925e-06,135.6,6.853,6.969,1908.8,462,,,,
+7920,1,0.03885,6.6843670348343244e-06,126.0,6.853,6.969,1913.9,458,,,,
+7940,1,0.01165,6.620645709430756e-06,142.8,6.853,6.969,1918.3,454,,,,
+7960,1,0.02126,6.5569243840271874e-06,195.0,6.853,6.969,1921.6,449,,,,
+7980,1,0.03034,6.493203058623619e-06,143.6,6.853,6.969,1926.1,445,,,,
+8000,1,0.01198,6.429481733220051e-06,154.7,6.853,6.969,1930.2,440,,,,
+8020,1,0.02995,6.365760407816483e-06,142.9,6.853,6.969,1934.7,436,,,,
+8040,1,0.00352,6.302039082412914e-06,145.8,6.853,6.969,1939.1,432,,,,
+8060,1,0.01366,6.238317757009346e-06,142.9,6.853,6.969,1943.6,427,,,,
+8080,1,0.01003,6.174596431605777e-06,137.5,6.853,6.969,1948.2,423,,,,
+8100,1,0.02186,6.110875106202209e-06,121.1,6.853,6.969,1953.5,419,,,,
+8120,1,0.02257,6.04715378079864e-06,133.9,6.853,6.969,1958.3,415,,,,
+8140,1,0.02613,5.983432455395072e-06,142.2,6.853,6.969,1962.8,410,,,,
+8160,1,0.001,5.919711129991504e-06,125.1,6.853,6.969,1967.9,406,,,,
+8180,1,0.00948,5.855989804587936e-06,118.8,6.853,6.969,1973.3,402,,,,
+8200,1,0.03258,5.792268479184367e-06,120.2,6.853,6.969,1978.6,398,,,,
+8220,1,0.03071,5.728547153780799e-06,137.1,6.853,6.969,1983.3,394,,,,
+8240,1,0.02108,5.664825828377231e-06,131.0,6.853,6.969,1988.2,389,,,,
+8260,1,0.02716,5.601104502973663e-06,178.3,6.853,6.969,1991.8,385,,,,
+8280,1,0.00217,5.537383177570093e-06,191.8,6.853,6.969,1995.1,380,,,,
+8300,1,0.00801,5.473661852166525e-06,165.8,6.853,6.969,1998.9,376,,,,
+8320,1,0.03542,5.409940526762957e-06,141.3,6.853,6.969,2003.5,371,,,,
+8340,1,0.02096,5.346219201359388e-06,218.2,6.853,6.969,2006.4,367,,,,
+8360,1,0.00848,5.28249787595582e-06,130.1,6.853,6.969,2011.3,362,,,,
+8380,1,0.00668,5.218776550552252e-06,150.8,6.853,6.969,2015.6,358,,,,
+8400,1,0.00665,5.155055225148684e-06,151.3,6.853,6.969,2019.8,354,,,,
+8420,1,0.01872,5.091333899745115e-06,176.5,6.853,6.969,2023.4,349,,,,
+8440,1,0.0294,5.027612574341547e-06,153.8,6.853,6.969,2027.6,345,,,,
+8460,1,0.02063,4.9638912489379785e-06,189.1,6.853,6.969,2031.0,340,,,,
+8480,1,0.01562,4.9001699235344096e-06,130.6,6.853,6.969,2035.9,336,,,,
+8500,1,0.03736,4.836448598130841e-06,169.6,6.853,6.969,2039.7,331,,,,
+8520,1,0.01841,4.7727272727272725e-06,157.6,6.853,6.969,2043.7,327,,,,
+8540,1,0.01969,4.7090059473237045e-06,125.5,6.853,6.969,2048.8,323,,,,
+8560,1,0.0146,4.645284621920136e-06,149.4,6.853,6.969,2053.1,318,,,,
+8580,1,0.00593,4.5815632965165674e-06,126.9,6.853,6.969,2058.1,314,,,,
+8600,1,0.02436,4.517841971112999e-06,157.5,6.853,6.969,2062.2,310,,,,
+8620,1,0.03281,4.454120645709431e-06,143.0,6.853,6.969,2066.7,305,,,,
+8640,1,0.0067,4.390399320305863e-06,163.7,6.853,6.969,2070.6,301,,,,
+8660,1,0.02285,4.326677994902294e-06,134.1,6.853,6.969,2075.4,297,,,,
+8680,1,0.01469,4.262956669498725e-06,110.9,6.853,6.969,2081.1,292,,,,
+8700,1,0.00978,4.199235344095157e-06,154.5,6.853,6.969,2085.3,288,,,,
+8720,1,0.01917,4.135514018691588e-06,148.1,6.853,6.969,2089.6,284,,,,
+8740,1,0.00984,4.07179269328802e-06,164.8,6.853,6.969,2093.5,279,,,,
+8750,1,0.02021,4.039932030586237e-06,177.5,6.853,6.969,2095.3,277,0.02933,0.99235,0.99957,0.96639
diff --git a/presentation/checkpoints/finetune/ft-ce/epochs.csv b/presentation/checkpoints/finetune/ft-ce/epochs.csv
new file mode 100644
index 0000000000000000000000000000000000000000..33931f666939babf3031798cd783533604b07994
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-ce/epochs.csv
@@ -0,0 +1,7 @@
+epoch,step,is_epoch_end,n,val_loss,accuracy,precision,recall,f1,roc_auc,pr_auc,tn,fp,fn,tp,threshold,f1_optimal_threshold,f1_at_optimal_threshold,f1_unwanted,n_3way,accuracy_3way,macro_f1_3way,precision_legitimate,recall_legitimate,f1_legitimate,precision_phishing,recall_phishing,f1_phishing,precision_spam,recall_spam,f1_spam,cm3_ll,cm3_lp,cm3_ls,cm3_pl,cm3_pp,cm3_ps,cm3_sl,cm3_sp,cm3_ss,n_unwanted_3way,pvs_phish_as_phish,pvs_phish_as_spam,pvs_spam_as_phish,pvs_spam_as_spam,eval_seconds,is_best
+0,1250,0,15284,0.06192874319186482,0.9824653232138184,0.9774785510009533,0.9904612412460758,0.9839270720882811,0.9979492956819666,0.9981618955324122,6813,189,79,8203,0.5,0.483214795589447,0.9841117572995983,0.9839270720882811,8105,0.9738433066008637,0.9250813429234777,0.9982425307557118,0.9734361610968295,0.9856832971800434,0.7865707434052758,0.9647058823529412,0.8665785997357992,0.8709302325581395,0.981651376146789,0.9229821318545902,6816,84,102,3,328,9,9,5,749,1103,330,10,8,755,48.6,1
+0,2500,0,15284,0.03543492886477882,0.9892043967547762,0.9887992291942671,0.9913064477179425,0.9900512511305396,0.9991295997741978,0.999168824631493,6909,93,72,8210,0.5,0.31836315989494324,0.9903192832661897,0.9900512511305396,8105,0.9851943244910549,0.958326511483941,0.9985555395059945,0.9872893459011711,0.9928904847396769,0.9635258358662614,0.9323529411764706,0.9476831091180867,0.8851113716295428,0.9895150720838795,0.9344059405940595,6913,11,78,3,317,20,7,1,755,1103,318,22,1,762,47.4,1
+0,3750,0,15284,0.04511974923150822,0.9844935880659513,0.9966662550932214,0.974643805843999,0.9855320188022709,0.9994927967936301,0.9995710185256946,6975,27,210,8072,0.5,0.08305464684963226,0.991784462969675,0.9855320188022709,8105,0.9921036397285626,0.9774728666158626,0.9950085567598402,0.9964295915452728,0.9957185671471386,0.9567723342939481,0.9764705882352941,0.9665211062590975,0.9812332439678284,0.9593709043250328,0.9701789264413518,6977,11,14,8,332,0,27,4,732,1103,340,0,6,757,47.8,0
+0,5000,0,15284,0.027098098741997853,0.9923449358806595,0.9938308939155679,0.992030910408114,0.992930086410055,0.9995503578823617,0.9996107014519675,6951,51,66,8216,0.5,0.31750011444091797,0.9931808581256412,0.992930086410055,8105,0.9922270203578039,0.9781799660145108,0.9988504095416008,0.9927163667523564,0.995773941694721,0.9390581717451524,0.9970588235294118,0.9671897289586305,0.9579617834394905,0.9855832241153342,0.9715762273901809,6951,18,33,1,339,0,7,4,752,1103,340,0,4,759,47.7,1
+0,5009,1,15284,0.03684096241911429,0.9888118293640408,0.9820515868299061,0.9975851243660951,0.9897574123989218,0.9995484092894837,0.9996088693166196,6851,151,20,8262,0.5,0.9320008754730225,0.9931217569687462,0.9897574123989218,8105,0.9809993830968537,0.9570998230201134,0.999708284714119,0.9788631819480148,0.9891759272622312,0.935933147632312,0.9882352941176471,0.9613733905579399,0.8550561797752809,0.9973787680209698,0.9207501512401693,6854,22,126,1,336,3,1,1,761,1103,337,3,1,762,47.7,0
+1,6250,0,15284,0.03418207616666885,0.9908400942161738,0.9948942377826404,0.9881671093938662,0.9915192633874486,0.999114942217151,0.9989895693935168,6960,42,98,8184,0.5,0.0818609967827797,0.9928274365619916,0.9915192633874486,8105,0.9933374460209747,0.9813139522024249,0.9984220341414431,0.9940017137960583,0.9962069705861304,0.9492957746478873,0.9911764705882353,0.9697841726618706,0.9679075738125802,0.9882044560943644,0.9779507133592736,6960,17,25,3,337,0,8,1,754,1103,339,1,1,762,47.9,0
diff --git a/presentation/checkpoints/finetune/ft-ce/metrics.csv b/presentation/checkpoints/finetune/ft-ce/metrics.csv
new file mode 100644
index 0000000000000000000000000000000000000000..a544b7dc1657194344b96159498fe340cabdfeaa
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-ce/metrics.csv
@@ -0,0 +1,317 @@
+step,epoch,loss,lr,examples_per_s,max_alloc_gb,max_reserved_gb,elapsed_s,eta_s,val_loss,val_f1,val_roc_auc,val_macro_f1_3way
+20,0,0.85624,6.984478935698447e-07,59.4,12.957,13.131,10.8,8087,,,,
+40,0,0.70976,1.3636363636363636e-06,64.0,12.957,13.131,20.8,7787,,,,
+60,0,0.64808,2.0288248337028825e-06,82.5,12.957,13.131,28.5,7119,,,,
+80,0,0.50167,2.694013303769401e-06,92.4,12.957,13.131,35.5,6626,,,,
+100,0,0.4,3.3592017738359202e-06,81.2,12.957,13.131,43.3,6471,,,,
+120,0,0.32445,4.024390243902439e-06,86.2,12.957,13.131,50.8,6308,,,,
+140,0,0.27761,4.6895787139689575e-06,73.2,12.957,13.131,59.5,6329,,,,
+160,0,0.25885,5.354767184035477e-06,85.8,12.957,13.131,67.0,6224,,,,
+180,0,0.16945,6.019955654101996e-06,67.1,12.957,13.131,76.5,6312,,,,
+200,0,0.1589,6.685144124168515e-06,105.1,12.957,13.131,82.6,6124,,,,
+220,0,0.16955,7.350332594235034e-06,63.8,12.957,13.131,92.6,6235,,,,
+240,0,0.1565,8.015521064301553e-06,83.4,12.957,13.131,100.3,6180,,,,
+260,0,0.1664,8.680709534368071e-06,70.7,12.957,13.131,109.4,6211,,,,
+280,0,0.12506,9.34589800443459e-06,68.1,12.957,13.131,118.8,6254,,,,
+300,0,0.17675,1.0011086474501108e-05,70.8,12.957,13.131,127.8,6273,,,,
+320,0,0.13385,1.0676274944567628e-05,67.1,12.957,13.131,137.3,6312,,,,
+340,0,0.13462,1.1341463414634148e-05,66.3,12.957,13.131,147.0,6349,,,,
+360,0,0.09442,1.2006651884700666e-05,96.9,12.957,13.131,153.6,6258,,,,
+380,0,0.17428,1.2671840354767186e-05,82.0,12.957,13.131,161.4,6221,,,,
+400,0,0.09234,1.3337028824833704e-05,70.9,12.957,13.131,170.4,6232,,,,
+420,0,0.14473,1.4002217294900222e-05,89.4,12.957,13.131,177.6,6176,,,,
+440,0,0.10938,1.466740576496674e-05,63.5,12.957,13.131,187.7,6222,,,,
+460,0,0.09414,1.5332594235033262e-05,98.1,12.957,13.131,194.2,6150,,,,
+480,0,0.0984,1.599778270509978e-05,72.1,12.957,13.131,203.1,6155,,,,
+500,0,0.09021,1.66629711751663e-05,67.8,12.957,13.131,212.5,6175,,,,
+520,0,0.10547,1.732815964523282e-05,74.9,12.957,13.133,221.1,6167,,,,
+540,0,0.09993,1.7993348115299335e-05,81.7,12.957,13.133,228.9,6141,,,,
+560,0,0.07586,1.8658536585365855e-05,72.8,12.957,13.133,237.7,6141,,,,
+580,0,0.16722,1.9323725055432375e-05,69.1,12.957,13.133,247.0,6151,,,,
+600,0,0.11573,1.998891352549889e-05,76.6,12.957,13.133,255.3,6139,,,,
+620,0,0.11076,2.065410199556541e-05,92.3,12.957,13.133,262.2,6094,,,,
+640,0,0.1181,2.131929046563193e-05,93.2,12.957,13.133,269.1,6050,,,,
+660,0,0.05236,2.1984478935698448e-05,54.3,12.957,13.133,280.9,6115,,,,
+680,0,0.12206,2.2649667405764968e-05,73.2,12.957,13.133,289.7,6111,,,,
+700,0,0.10318,2.3314855875831488e-05,76.7,12.957,13.133,298.0,6099,,,,
+720,0,0.13271,2.3980044345898004e-05,57.7,12.957,13.133,308.7,6133,,,,
+740,0,0.07775,2.4645232815964524e-05,79.9,12.957,13.133,316.7,6114,,,,
+760,0,0.12985,2.531042128603104e-05,81.9,12.957,13.133,324.5,6091,,,,
+780,0,0.08311,2.597560975609756e-05,85.8,12.957,13.133,331.9,6063,,,,
+800,0,0.0931,2.664079822616408e-05,72.2,12.957,13.133,340.8,6061,,,,
+820,0,0.13663,2.7305986696230597e-05,67.8,12.957,13.133,350.2,6068,,,,
+840,0,0.08755,2.7971175166297117e-05,66.6,12.957,13.133,359.9,6078,,,,
+860,0,0.07562,2.8636363636363637e-05,91.8,12.957,13.133,366.8,6043,,,,
+880,0,0.08203,2.9301552106430153e-05,72.2,12.957,13.133,375.7,6040,,,,
+900,0,0.12903,2.9966740576496673e-05,73.9,12.957,13.133,384.4,6033,,,,
+920,0,0.07784,2.9961769911504427e-05,98.5,12.957,13.133,390.9,5993,,,,
+940,0,0.10167,2.991929203539823e-05,71.6,12.957,13.133,399.8,5991,,,,
+960,0,0.06448,2.9876814159292037e-05,80.7,12.957,13.133,407.7,5974,,,,
+980,0,0.19386,2.983433628318584e-05,80.5,12.957,13.133,415.7,5958,,,,
+1000,0,0.11475,2.9791858407079644e-05,66.6,12.957,13.133,425.3,5965,,,,
+1020,0,0.0955,2.9749380530973454e-05,71.8,12.957,13.133,434.2,5962,,,,
+1040,0,0.10636,2.9706902654867257e-05,67.7,12.957,13.133,443.6,5966,,,,
+1060,0,0.05161,2.9664424778761064e-05,74.8,12.957,13.133,452.2,5958,,,,
+1080,0,0.09595,2.9621946902654868e-05,63.0,12.957,13.133,462.3,5971,,,,
+1100,0,0.08516,2.9579469026548674e-05,70.8,12.957,13.133,471.4,5968,,,,
+1120,0,0.055,2.9536991150442478e-05,72.8,12.957,13.133,480.2,5962,,,,
+1140,0,0.11511,2.9494513274336285e-05,75.5,12.957,13.133,488.7,5953,,,,
+1160,0,0.07698,2.9452035398230088e-05,84.1,12.957,13.133,496.3,5933,,,,
+1180,0,0.06861,2.940955752212389e-05,86.3,12.957,13.133,503.7,5911,,,,
+1200,0,0.09102,2.93670796460177e-05,77.5,12.957,13.133,511.9,5899,,,,
+1220,0,0.07275,2.9324601769911505e-05,85.2,12.957,13.133,519.5,5879,,,,
+1240,0,0.04949,2.9282123893805312e-05,62.0,12.957,13.133,529.8,5890,,,,
+1250,0,0.11906,2.9260884955752214e-05,94.4,12.957,13.133,533.2,5876,0.06193,0.98393,0.99795,0.92508
+1260,0,0.05512,2.9239646017699115e-05,96.0,12.957,13.133,591.0,5862,,,,
+1280,0,0.08578,2.9197168141592922e-05,63.4,12.957,13.133,601.1,5870,,,,
+1300,0,0.10406,2.9154690265486726e-05,74.3,12.957,13.133,609.7,5863,,,,
+1320,0,0.11267,2.9112212389380533e-05,68.5,12.957,13.133,619.0,5862,,,,
+1340,0,0.06352,2.9069734513274336e-05,79.5,12.957,13.133,627.1,5849,,,,
+1360,0,0.06484,2.9027256637168143e-05,67.6,12.957,13.133,636.5,5849,,,,
+1380,0,0.06628,2.8984778761061946e-05,69.3,12.957,13.133,645.8,5848,,,,
+1400,0,0.07169,2.894230088495575e-05,65.6,12.957,13.133,655.5,5850,,,,
+1420,0,0.06728,2.889982300884956e-05,71.2,12.957,13.133,664.5,5846,,,,
+1440,0,0.0795,2.8857345132743363e-05,67.7,12.957,13.133,674.0,5845,,,,
+1460,0,0.09554,2.881486725663717e-05,62.8,12.957,13.133,684.2,5851,,,,
+1480,0,0.05519,2.8772389380530974e-05,67.9,12.957,13.133,693.6,5850,,,,
+1500,0,0.05006,2.872991150442478e-05,76.1,12.957,13.133,702.0,5839,,,,
+1520,0,0.08117,2.8687433628318584e-05,75.1,12.957,13.133,710.5,5830,,,,
+1540,0,0.06642,2.864495575221239e-05,75.4,12.957,13.133,719.0,5820,,,,
+1560,0,0.06205,2.8602477876106194e-05,82.3,12.957,13.133,726.8,5804,,,,
+1580,0,0.06812,2.856e-05,63.9,12.957,13.133,736.8,5807,,,,
+1600,0,0.07865,2.8517522123893804e-05,74.7,12.957,13.133,745.4,5798,,,,
+1620,0,0.05096,2.847504424778761e-05,63.8,12.957,13.133,755.4,5801,,,,
+1640,0,0.09353,2.8432566371681418e-05,71.3,12.957,13.133,764.4,5795,,,,
+1660,0,0.06362,2.839008849557522e-05,77.3,12.957,13.133,772.6,5783,,,,
+1680,0,0.06525,2.834761061946903e-05,62.8,12.957,13.133,782.8,5787,,,,
+1700,0,0.07314,2.8305132743362832e-05,59.1,12.957,13.133,793.7,5795,,,,
+1720,0,0.05784,2.826265486725664e-05,72.6,12.957,13.133,802.5,5787,,,,
+1740,0,0.06446,2.8220176991150442e-05,70.3,12.957,13.133,811.6,5781,,,,
+1760,0,0.05686,2.817769911504425e-05,67.7,12.957,13.133,821.0,5778,,,,
+1780,0,0.04756,2.8135221238938052e-05,77.3,12.957,13.133,829.3,5766,,,,
+1800,0,0.03991,2.809274336283186e-05,72.2,12.957,13.133,838.2,5759,,,,
+1820,0,0.07446,2.8050265486725666e-05,73.9,12.957,13.133,846.8,5750,,,,
+1840,0,0.05358,2.800778761061947e-05,96.4,12.957,13.133,853.5,5726,,,,
+1860,0,0.05461,2.7965309734513276e-05,79.1,12.957,13.133,861.6,5714,,,,
+1880,0,0.0674,2.792283185840708e-05,58.8,12.957,13.133,872.5,5720,,,,
+1900,0,0.0564,2.7880353982300886e-05,77.4,12.957,13.133,880.7,5709,,,,
+1920,0,0.06604,2.783787610619469e-05,68.4,12.957,13.133,890.1,5704,,,,
+1940,0,0.04758,2.7795398230088497e-05,81.8,12.957,13.133,897.9,5690,,,,
+1960,0,0.05814,2.77529203539823e-05,90.8,12.957,13.133,905.0,5670,,,,
+1980,0,0.081,2.7710442477876107e-05,67.5,12.957,13.133,914.4,5667,,,,
+2000,0,0.03899,2.766796460176991e-05,71.7,12.957,13.133,923.4,5660,,,,
+2020,0,0.06392,2.7625486725663717e-05,66.5,12.957,13.133,933.0,5657,,,,
+2040,0,0.06523,2.7583008849557524e-05,71.9,12.957,13.133,941.9,5649,,,,
+2060,0,0.03831,2.7540530973451328e-05,63.0,12.957,13.133,952.1,5650,,,,
+2080,0,0.08336,2.7498053097345134e-05,63.0,12.957,13.133,962.2,5650,,,,
+2100,0,0.04955,2.7455575221238938e-05,67.7,12.957,13.133,971.7,5646,,,,
+2120,0,0.05433,2.7413097345132745e-05,63.2,12.957,13.133,981.8,5646,,,,
+2140,0,0.08315,2.7370619469026548e-05,54.5,12.957,13.133,993.5,5655,,,,
+2160,0,0.04532,2.7328141592920355e-05,68.7,12.957,13.133,1002.8,5649,,,,
+2180,0,0.07227,2.728566371681416e-05,74.5,12.957,13.133,1011.4,5639,,,,
+2200,0,0.02863,2.7243185840707965e-05,75.9,12.957,13.133,1019.9,5629,,,,
+2220,0,0.07338,2.7200707964601772e-05,61.3,12.957,13.133,1030.3,5629,,,,
+2240,0,0.05404,2.7158230088495575e-05,80.2,12.957,13.133,1038.3,5616,,,,
+2260,0,0.05889,2.7115752212389382e-05,64.2,12.957,13.133,1048.3,5614,,,,
+2280,0,0.10184,2.7073274336283186e-05,62.1,12.957,13.133,1058.6,5614,,,,
+2300,0,0.04184,2.7030796460176993e-05,70.1,12.957,13.133,1067.7,5607,,,,
+2320,0,0.056,2.6988318584070796e-05,73.1,12.957,13.133,1076.4,5598,,,,
+2340,0,0.02039,2.6945840707964603e-05,82.9,12.957,13.133,1084.2,5583,,,,
+2360,0,0.05052,2.6903362831858406e-05,62.9,12.957,13.133,1094.3,5581,,,,
+2380,0,0.0275,2.6860884955752213e-05,74.9,12.957,13.133,1102.9,5571,,,,
+2400,0,0.04963,2.6818407079646016e-05,78.1,12.957,13.133,1111.1,5559,,,,
+2420,0,0.0751,2.6775929203539827e-05,83.7,12.957,13.133,1118.7,5544,,,,
+2440,0,0.03768,2.673345132743363e-05,82.3,12.957,13.133,1126.5,5530,,,,
+2460,0,0.08904,2.6690973451327434e-05,63.4,12.957,13.133,1136.6,5528,,,,
+2480,0,0.0475,2.664849557522124e-05,64.0,12.957,13.133,1146.6,5525,,,,
+2500,0,0.04786,2.6606017699115044e-05,66.8,12.957,13.133,1156.2,5520,0.03543,0.99005,0.99913,0.95833
+2520,0,0.05126,2.656353982300885e-05,70.2,12.957,13.133,1217.6,5513,,,,
+2540,0,0.03888,2.6521061946902654e-05,64.9,12.957,13.133,1227.5,5510,,,,
+2560,0,0.07304,2.647858407079646e-05,66.1,12.957,13.133,1237.2,5505,,,,
+2580,0,0.04437,2.6436106194690264e-05,83.9,12.957,13.133,1244.8,5490,,,,
+2600,0,0.0518,2.639362831858407e-05,68.6,12.957,13.133,1254.1,5484,,,,
+2620,0,0.06365,2.6351150442477878e-05,65.5,12.957,13.133,1263.9,5479,,,,
+2640,0,0.02907,2.6308672566371685e-05,78.7,12.957,13.133,1272.0,5467,,,,
+2660,0,0.048,2.6266194690265488e-05,89.6,12.957,13.133,1279.2,5451,,,,
+2680,0,0.05395,2.6223716814159292e-05,73.7,12.957,13.133,1287.9,5441,,,,
+2700,0,0.0607,2.61812389380531e-05,64.9,12.957,13.133,1297.7,5437,,,,
+2720,0,0.04612,2.6138761061946902e-05,79.0,12.957,13.133,1305.8,5425,,,,
+2740,0,0.06064,2.609628318584071e-05,66.7,12.957,13.133,1315.4,5420,,,,
+2760,0,0.04957,2.6053805309734512e-05,69.6,12.957,13.133,1324.6,5413,,,,
+2780,0,0.04937,2.601132743362832e-05,73.3,12.957,13.133,1333.4,5403,,,,
+2800,0,0.06199,2.5968849557522123e-05,60.9,12.957,13.133,1343.9,5402,,,,
+2820,0,0.02124,2.5926371681415933e-05,86.2,12.957,13.133,1351.3,5387,,,,
+2840,0,0.07021,2.5883893805309736e-05,59.7,12.957,13.133,1362.0,5386,,,,
+2860,0,0.07272,2.584141592920354e-05,62.1,12.957,13.133,1372.3,5384,,,,
+2880,0,0.0396,2.5798938053097346e-05,75.4,12.957,13.133,1380.8,5373,,,,
+2900,0,0.03616,2.575646017699115e-05,79.0,12.957,13.133,1388.9,5361,,,,
+2920,0,0.05072,2.5713982300884957e-05,78.9,12.957,13.133,1397.0,5350,,,,
+2940,0,0.06447,2.567150442477876e-05,61.5,12.957,13.133,1407.4,5347,,,,
+2960,0,0.04205,2.5629026548672567e-05,77.5,12.957,13.133,1415.7,5336,,,,
+2980,0,0.01675,2.558654867256637e-05,94.1,12.957,13.133,1422.5,5319,,,,
+3000,0,0.04313,2.5544070796460177e-05,67.8,12.957,13.133,1431.9,5312,,,,
+3020,0,0.02565,2.5501592920353984e-05,88.5,12.957,13.133,1439.2,5297,,,,
+3040,0,0.05681,2.545911504424779e-05,77.1,12.957,13.133,1447.5,5286,,,,
+3060,0,0.0312,2.5416637168141594e-05,80.9,12.957,13.133,1455.4,5274,,,,
+3080,0,0.08292,2.5374159292035398e-05,67.8,12.957,13.133,1464.8,5268,,,,
+3100,0,0.04439,2.5331681415929205e-05,66.2,12.957,13.133,1474.5,5262,,,,
+3120,0,0.04294,2.5289203539823008e-05,71.1,12.957,13.133,1483.5,5254,,,,
+3140,0,0.05001,2.5246725663716815e-05,70.2,12.957,13.133,1492.6,5246,,,,
+3160,0,0.03482,2.5204247787610618e-05,82.4,12.957,13.133,1500.4,5233,,,,
+3180,0,0.03051,2.5161769911504425e-05,70.9,12.957,13.133,1509.4,5225,,,,
+3200,0,0.07109,2.511929203539823e-05,69.7,12.957,13.133,1518.6,5218,,,,
+3220,0,0.05198,2.507681415929204e-05,73.9,12.957,13.133,1527.2,5208,,,,
+3240,0,0.01715,2.5034336283185842e-05,68.3,12.957,13.133,1536.6,5202,,,,
+3260,0,0.04018,2.499185840707965e-05,69.8,12.957,13.133,1545.8,5194,,,,
+3280,0,0.02863,2.4949380530973452e-05,65.9,12.957,13.133,1555.5,5188,,,,
+3300,0,0.0823,2.4906902654867256e-05,86.1,12.957,13.133,1562.9,5175,,,,
+3320,0,0.03567,2.4864424778761063e-05,73.8,12.957,13.133,1571.6,5165,,,,
+3340,0,0.04609,2.4821946902654866e-05,101.6,12.957,13.133,1577.9,5147,,,,
+3360,0,0.0212,2.4779469026548673e-05,66.8,12.957,13.133,1587.5,5141,,,,
+3380,0,0.05717,2.4736991150442476e-05,67.9,12.957,13.133,1596.9,5135,,,,
+3400,0,0.08482,2.4694513274336283e-05,80.5,12.957,13.133,1604.8,5123,,,,
+3420,0,0.05121,2.465203539823009e-05,73.3,12.957,13.133,1613.6,5114,,,,
+3440,0,0.05245,2.4609557522123897e-05,68.7,12.957,13.133,1622.9,5107,,,,
+3460,0,0.04962,2.45670796460177e-05,73.6,12.957,13.133,1631.6,5097,,,,
+3480,0,0.05532,2.4524601769911507e-05,58.0,12.957,13.133,1642.6,5096,,,,
+3500,0,0.04682,2.448212389380531e-05,85.4,12.957,13.133,1650.1,5083,,,,
+3520,0,0.02012,2.4439646017699114e-05,96.1,12.957,13.133,1656.8,5067,,,,
+3540,0,0.06732,2.439716814159292e-05,77.4,12.957,13.133,1665.0,5056,,,,
+3560,0,0.03927,2.4354690265486724e-05,80.2,12.957,13.133,1673.0,5045,,,,
+3580,0,0.04939,2.431221238938053e-05,75.6,12.957,13.133,1681.5,5035,,,,
+3600,0,0.05605,2.4269734513274335e-05,67.3,12.957,13.133,1691.0,5029,,,,
+3620,0,0.05469,2.4227256637168145e-05,69.7,12.957,13.133,1700.2,5021,,,,
+3640,0,0.03765,2.4184778761061948e-05,96.2,12.957,13.133,1706.8,5005,,,,
+3660,0,0.05929,2.4142300884955755e-05,76.1,12.957,13.133,1715.2,4995,,,,
+3680,0,0.04146,2.409982300884956e-05,69.4,12.957,13.133,1724.5,4988,,,,
+3700,0,0.02759,2.4057345132743362e-05,86.4,12.957,13.133,1731.9,4975,,,,
+3720,0,0.03002,2.401486725663717e-05,69.8,12.957,13.133,1741.0,4967,,,,
+3740,0,0.05959,2.3972389380530972e-05,68.2,12.957,13.133,1750.4,4960,,,,
+3750,0,0.06421,2.3951150442477877e-05,76.0,12.957,13.133,1754.6,4955,0.04512,0.98553,0.99949,0.97747
+3760,0,0.026,2.392991150442478e-05,72.9,12.957,13.133,1810.8,4951,,,,
+3780,0,0.03044,2.3887433628318582e-05,76.9,12.957,13.133,1819.2,4941,,,,
+3800,0,0.05317,2.384495575221239e-05,73.6,12.957,13.133,1827.9,4932,,,,
+3820,0,0.05224,2.3802477876106196e-05,70.5,12.957,13.133,1836.9,4924,,,,
+3840,0,0.05099,2.3760000000000003e-05,78.8,12.957,13.133,1845.1,4913,,,,
+3860,0,0.0386,2.3717522123893806e-05,68.5,12.957,13.133,1854.4,4906,,,,
+3880,0,0.0466,2.3675044247787613e-05,76.2,12.957,13.133,1862.8,4896,,,,
+3900,0,0.03891,2.3632566371681417e-05,72.0,12.957,13.133,1871.7,4888,,,,
+3920,0,0.04601,2.359008849557522e-05,71.7,12.957,13.133,1880.6,4879,,,,
+3940,0,0.06498,2.3547610619469027e-05,61.3,12.957,13.133,1891.1,4875,,,,
+3960,0,0.0367,2.350513274336283e-05,77.0,12.957,13.133,1899.4,4865,,,,
+3980,0,0.08285,2.3462654867256637e-05,84.9,12.957,13.133,1906.9,4853,,,,
+4000,0,0.00842,2.342017699115044e-05,69.2,12.957,13.133,1916.2,4845,,,,
+4020,0,0.04797,2.337769911504425e-05,71.9,12.957,13.133,1925.1,4837,,,,
+4040,0,0.04773,2.3335221238938054e-05,64.9,12.957,13.133,1934.9,4831,,,,
+4060,0,0.04125,2.329274336283186e-05,76.3,12.957,13.133,1943.3,4821,,,,
+4080,0,0.04833,2.3250265486725664e-05,59.3,12.957,13.133,1954.1,4817,,,,
+4100,0,0.02193,2.320778761061947e-05,73.7,12.957,13.133,1962.8,4808,,,,
+4120,0,0.04386,2.3165309734513275e-05,74.7,12.957,13.133,1971.4,4799,,,,
+4140,0,0.05543,2.3122831858407078e-05,66.1,12.957,13.133,1981.0,4792,,,,
+4160,0,0.04602,2.3080353982300885e-05,70.6,12.957,13.133,1990.1,4784,,,,
+4180,0,0.05736,2.303787610619469e-05,65.1,12.957,13.133,1999.9,4778,,,,
+4200,0,0.03448,2.2995398230088495e-05,74.3,12.957,13.133,2008.6,4769,,,,
+4220,0,0.01829,2.2952920353982302e-05,96.5,12.957,13.133,2015.2,4754,,,,
+4240,0,0.0418,2.291044247787611e-05,90.1,12.957,13.133,2022.3,4741,,,,
+4260,0,0.05417,2.2867964601769912e-05,59.3,12.957,13.133,2033.1,4738,,,,
+4280,0,0.02891,2.282548672566372e-05,76.9,12.957,13.133,2041.4,4728,,,,
+4300,0,0.03052,2.2783008849557523e-05,60.5,12.957,13.133,2052.0,4723,,,,
+4320,0,0.03922,2.274053097345133e-05,81.6,12.957,13.133,2059.8,4712,,,,
+4340,0,0.0154,2.2698053097345133e-05,63.5,12.957,13.133,2069.9,4706,,,,
+4360,0,0.05141,2.2655575221238936e-05,61.9,12.957,13.133,2080.2,4701,,,,
+4380,0,0.01486,2.2613097345132743e-05,65.1,12.957,13.133,2090.1,4695,,,,
+4400,0,0.0768,2.2570619469026547e-05,62.6,12.957,13.133,2100.3,4690,,,,
+4420,0,0.04374,2.2528141592920357e-05,69.3,12.957,13.133,2109.5,4682,,,,
+4440,0,0.02797,2.248566371681416e-05,81.8,12.957,13.133,2117.4,4670,,,,
+4460,0,0.02042,2.2443185840707967e-05,71.2,12.957,13.133,2126.3,4662,,,,
+4480,0,0.05212,2.240070796460177e-05,65.1,12.957,13.133,2136.2,4656,,,,
+4500,0,0.02174,2.2358230088495577e-05,74.1,12.957,13.133,2144.8,4646,,,,
+4520,0,0.0328,2.231575221238938e-05,68.2,12.957,13.133,2154.2,4639,,,,
+4540,0,0.02082,2.2273274336283184e-05,78.4,12.957,13.133,2162.4,4628,,,,
+4560,0,0.02911,2.223079646017699e-05,60.9,12.957,13.133,2172.9,4623,,,,
+4580,0,0.03505,2.2188318584070794e-05,62.9,12.957,13.133,2183.1,4618,,,,
+4600,0,0.0349,2.21458407079646e-05,62.4,12.957,13.133,2193.3,4612,,,,
+4620,0,0.03229,2.2103362831858408e-05,67.7,12.957,13.133,2202.8,4605,,,,
+4640,0,0.03537,2.2060884955752215e-05,98.2,12.957,13.133,2209.3,4591,,,,
+4660,0,0.07475,2.201840707964602e-05,64.6,12.957,13.133,2219.2,4584,,,,
+4680,0,0.02508,2.1975929203539825e-05,76.1,12.957,13.133,2227.6,4574,,,,
+4700,0,0.00742,2.193345132743363e-05,80.0,12.957,13.133,2235.6,4564,,,,
+4720,0,0.05772,2.1890973451327435e-05,63.1,12.957,13.133,2245.8,4558,,,,
+4740,0,0.01976,2.184849557522124e-05,70.9,12.957,13.133,2254.8,4549,,,,
+4760,0,0.0855,2.1806017699115042e-05,70.6,12.957,13.133,2263.8,4541,,,,
+4780,0,0.04749,2.176353982300885e-05,84.5,12.957,13.133,2271.4,4529,,,,
+4800,0,0.02546,2.1721061946902653e-05,72.5,12.957,13.133,2280.2,4520,,,,
+4820,0,0.02738,2.1678584070796463e-05,71.4,12.957,13.133,2289.2,4512,,,,
+4840,0,0.0293,2.1636106194690266e-05,58.1,12.957,13.133,2300.2,4507,,,,
+4860,0,0.02421,2.1593628318584073e-05,82.9,12.957,13.133,2307.9,4496,,,,
+4880,0,0.02581,2.1551150442477877e-05,74.0,12.957,13.133,2316.6,4487,,,,
+4900,0,0.02588,2.1508672566371683e-05,62.2,12.957,13.133,2326.9,4481,,,,
+4920,0,0.03799,2.1466194690265487e-05,66.6,12.957,13.133,2336.5,4474,,,,
+4940,0,0.01155,2.1423716814159294e-05,71.9,12.957,13.133,2345.4,4465,,,,
+4960,0,0.0244,2.1381238938053097e-05,62.4,12.957,13.133,2355.6,4459,,,,
+4980,0,0.07611,2.13387610619469e-05,64.9,12.957,13.133,2365.5,4452,,,,
+5000,0,0.03071,2.1296283185840707e-05,71.9,12.957,13.133,2374.4,4443,0.0271,0.99293,0.99955,0.97818
+5009,0,0.08859,2.127716814159292e-05,96.9,12.957,13.133,2431.6,4437,0.03684,0.98976,0.99955,0.9571
+5020,1,0.03024,2.1253805309734514e-05,81.7,12.957,13.133,2487.8,4431,,,,
+5040,1,0.02892,2.121132743362832e-05,75.4,12.957,13.133,2496.3,4422,,,,
+5060,1,0.02383,2.1168849557522124e-05,73.7,12.957,13.133,2505.0,4413,,,,
+5080,1,0.02632,2.112637168141593e-05,63.8,12.957,13.133,2515.0,4406,,,,
+5100,1,0.04045,2.1083893805309735e-05,86.6,12.957,13.133,2522.4,4394,,,,
+5120,1,0.03004,2.104141592920354e-05,76.2,12.957,13.133,2530.8,4385,,,,
+5140,1,0.0296,2.0998938053097345e-05,67.4,12.957,13.133,2540.3,4377,,,,
+5160,1,0.01027,2.0956460176991152e-05,82.0,12.957,13.133,2548.1,4366,,,,
+5180,1,0.00286,2.0913982300884955e-05,108.6,12.957,13.133,2554.0,4352,,,,
+5200,1,0.02711,2.087150442477876e-05,68.2,12.957,13.133,2563.4,4344,,,,
+5220,1,0.00229,2.082902654867257e-05,81.3,12.957,13.133,2571.3,4333,,,,
+5240,1,0.0214,2.0786548672566372e-05,62.2,12.957,13.133,2581.5,4327,,,,
+5260,1,0.02366,2.074407079646018e-05,75.0,12.957,13.133,2590.1,4318,,,,
+5280,1,0.01492,2.0701592920353983e-05,72.5,12.957,13.133,2598.9,4309,,,,
+5300,1,0.01082,2.065911504424779e-05,61.6,12.957,13.133,2609.3,4303,,,,
+5320,1,0.01411,2.0616637168141593e-05,57.7,12.957,13.133,2620.4,4298,,,,
+5340,1,0.04952,2.05741592920354e-05,71.4,12.957,13.133,2629.4,4289,,,,
+5360,1,0.01072,2.0531681415929203e-05,71.8,12.957,13.133,2638.3,4281,,,,
+5380,1,0.01244,2.0489203539823007e-05,62.6,12.957,13.133,2648.5,4274,,,,
+5400,1,0.02213,2.0446725663716813e-05,62.6,12.957,13.133,2658.7,4268,,,,
+5420,1,0.02963,2.040424778761062e-05,67.5,12.957,13.133,2668.2,4260,,,,
+5440,1,0.05407,2.0361769911504427e-05,67.3,12.957,13.133,2677.7,4252,,,,
+5460,1,0.05106,2.031929203539823e-05,67.7,12.957,13.133,2687.2,4245,,,,
+5480,1,0.0098,2.0276814159292037e-05,67.3,12.957,13.133,2696.7,4237,,,,
+5500,1,0.04684,2.023433628318584e-05,73.5,12.957,13.133,2705.4,4228,,,,
+5520,1,0.02262,2.0191858407079648e-05,86.0,12.957,13.133,2712.8,4216,,,,
+5540,1,0.01165,2.014938053097345e-05,61.1,12.957,13.133,2723.3,4210,,,,
+5560,1,0.03457,2.0106902654867258e-05,80.1,12.957,13.133,2731.3,4200,,,,
+5580,1,0.02719,2.006442477876106e-05,126.7,12.957,13.133,2736.4,4184,,,,
+5600,1,0.01925,2.0021946902654865e-05,83.2,12.957,13.133,2744.0,4174,,,,
+5620,1,0.01367,1.9979469026548675e-05,71.2,12.957,13.133,2753.0,4165,,,,
+5640,1,0.00484,1.993699115044248e-05,74.1,12.957,13.133,2761.7,4156,,,,
+5660,1,0.03598,1.9894513274336285e-05,64.5,12.957,13.133,2771.6,4149,,,,
+5680,1,0.02128,1.985203539823009e-05,73.8,12.957,13.133,2780.3,4139,,,,
+5700,1,0.04686,1.9809557522123895e-05,68.3,12.957,13.133,2789.6,4131,,,,
+5720,1,0.01992,1.97670796460177e-05,78.5,12.957,13.133,2797.8,4121,,,,
+5740,1,0.02711,1.9724601769911506e-05,88.8,12.957,13.133,2805.0,4110,,,,
+5760,1,0.00534,1.968212389380531e-05,64.9,12.957,13.133,2814.9,4103,,,,
+5780,1,0.05113,1.9639646017699116e-05,64.9,12.957,13.133,2824.7,4095,,,,
+5800,1,0.02856,1.959716814159292e-05,78.8,12.957,13.133,2832.8,4085,,,,
+5820,1,0.01213,1.9554690265486726e-05,60.3,12.957,13.133,2843.5,4079,,,,
+5840,1,0.0219,1.9512212389380533e-05,70.6,12.957,13.133,2852.5,4071,,,,
+5860,1,0.01937,1.9469734513274336e-05,71.1,12.957,13.133,2861.5,4062,,,,
+5880,1,0.0281,1.9427256637168143e-05,62.8,12.957,13.133,2871.7,4055,,,,
+5900,1,0.02451,1.9384778761061947e-05,70.6,12.957,13.133,2880.8,4047,,,,
+5920,1,0.0212,1.9342300884955754e-05,65.9,12.957,13.133,2890.5,4039,,,,
+5940,1,0.02897,1.9299823008849557e-05,65.9,12.957,13.133,2900.2,4032,,,,
+5960,1,0.02687,1.9257345132743364e-05,63.9,12.957,13.133,2910.2,4025,,,,
+5980,1,0.03089,1.9214867256637167e-05,66.3,12.957,13.133,2919.9,4017,,,,
+6000,1,0.01829,1.9172389380530974e-05,62.0,12.957,13.133,2930.2,4010,,,,
+6020,1,0.01982,1.912991150442478e-05,76.4,12.957,13.133,2938.6,4000,,,,
+6040,1,0.00795,1.9087433628318584e-05,76.2,12.957,13.133,2947.0,3991,,,,
+6060,1,0.00636,1.904495575221239e-05,63.0,12.957,13.133,2957.1,3984,,,,
+6080,1,0.02269,1.9002477876106195e-05,80.5,12.957,13.133,2965.1,3974,,,,
+6100,1,0.02266,1.896e-05,65.7,12.957,13.133,2974.8,3966,,,,
+6120,1,0.00613,1.8917522123893805e-05,72.9,12.957,13.133,2983.6,3957,,,,
+6140,1,0.02675,1.887504424778761e-05,93.3,12.957,13.133,2990.5,3945,,,,
+6160,1,0.0058,1.8832566371681415e-05,65.7,12.957,13.133,3000.2,3938,,,,
+6180,1,0.02163,1.8790088495575222e-05,70.1,12.957,13.133,3009.3,3929,,,,
+6200,1,0.01771,1.8747610619469025e-05,69.7,12.957,13.133,3018.5,3921,,,,
+6220,1,0.01566,1.8705132743362832e-05,78.8,12.957,13.133,3026.7,3911,,,,
+6240,1,0.04004,1.866265486725664e-05,80.5,12.957,13.133,3034.6,3900,,,,
+6250,1,0.02779,1.864141592920354e-05,47.7,12.957,13.133,3041.3,3899,0.03418,0.99152,0.99911,0.98131
diff --git a/presentation/checkpoints/finetune/ft-focal/epochs.csv b/presentation/checkpoints/finetune/ft-focal/epochs.csv
new file mode 100644
index 0000000000000000000000000000000000000000..efdd3c055738e20e8b0fbb2dc3d0ac89206c554c
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-focal/epochs.csv
@@ -0,0 +1,11 @@
+epoch,step,is_epoch_end,n,val_loss,accuracy,precision,recall,f1,roc_auc,pr_auc,tn,fp,fn,tp,threshold,f1_optimal_threshold,f1_at_optimal_threshold,f1_unwanted,n_3way,accuracy_3way,macro_f1_3way,precision_legitimate,recall_legitimate,f1_legitimate,precision_phishing,recall_phishing,f1_phishing,precision_spam,recall_spam,f1_spam,cm3_ll,cm3_lp,cm3_ls,cm3_pl,cm3_pp,cm3_ps,cm3_sl,cm3_sp,cm3_ss,n_unwanted_3way,pvs_phish_as_phish,pvs_phish_as_spam,pvs_spam_as_phish,pvs_spam_as_spam,eval_seconds,is_best
+0,1250,0,15284,0.11708931720434637,0.9839047369798483,0.9837466891403804,0.986597440231828,0.9851700024113818,0.9982902908135193,0.9983547392775961,6867,135,111,8171,0.5,0.4708244800567627,0.9856850715746421,0.9851700024113818,8105,0.9802590993214065,0.9496755413101702,0.9963799594555459,0.9827192230791203,0.9895024446361806,0.9748427672955975,0.9117647058823529,0.9422492401215805,0.8558456299659478,0.9882044560943644,0.9172749391727494,6881,8,113,16,310,14,9,0,754,1103,320,20,0,763,48.8,1
+0,2500,0,15284,0.04746348264017083,0.9888772572624968,0.986214337089427,0.9933590920067616,0.9897738209817132,0.9992167173956094,0.999243306710704,6887,115,55,8227,0.5,0.5908242464065552,0.9902256546397973,0.9897738209817132,8105,0.9854410857495374,0.9616931571319361,0.9986971627099016,0.9852899171665238,0.9919482386772106,0.9105691056910569,0.9882352941176471,0.9478138222849083,0.9082125603864735,0.9855832241153342,0.9453174104336896,6899,30,73,1,336,3,8,3,752,1103,337,3,3,760,47.8,1
+0,3750,0,15284,0.05659857270637129,0.9901203873331589,0.9957322277771004,0.9859937213233518,0.9908390462901171,0.9995619977070752,0.9996193211418505,6967,35,116,8166,0.5,0.3836457431316376,0.992391304347826,0.9908390462901171,8105,0.9938309685379395,0.9835704438976342,0.9975676062383746,0.9957155098543273,0.9966406975913087,0.9794117647058823,0.9794117647058823,0.9794117647058823,0.9664948453608248,0.9829619921363041,0.9746588693957114,6972,6,24,5,333,2,12,1,750,1103,337,3,1,762,48.1,1
+0,5000,0,15284,0.045222507490908344,0.9927375032713949,0.9924068940580932,0.9942042984786283,0.9933047831594185,0.9995788797639561,0.9996310389186062,6939,63,48,8234,0.5,0.46049919724464417,0.9933710979872242,0.9933047831594185,8105,0.9921036397285626,0.9796676498352132,0.9994242947610823,0.9917166523850328,0.9955555555555555,0.952247191011236,0.9970588235294118,0.9741379310344828,0.9463171036204744,0.9934469200524246,0.969309462915601,6944,15,43,1,339,0,3,2,758,1103,340,0,2,761,48.1,1
+0,5009,1,15284,0.05172481040965821,0.9928029311698509,0.9906340057636888,0.9961361989857522,0.9933774834437086,0.9995948478790446,0.9996407988938995,6924,78,32,8250,0.5,0.5638164281845093,0.993849493487699,0.9933774834437086,8105,0.9904996915484269,0.975949838644307,0.9994232987312572,0.9900028563267638,0.9946907734251687,0.9519774011299436,0.9911764705882353,0.9711815561959655,0.9312883435582822,0.9947575360419397,0.9619771863117871,6932,16,54,1,337,2,3,1,759,1103,338,2,2,761,47.8,1
+1,6250,0,15284,0.03721452226535215,0.9938497775451453,0.9949226305609284,0.9937213233518474,0.994321614111393,0.9997740666912638,0.9998119837761761,6960,42,52,8230,0.5,0.4559151530265808,0.9948098974049487,0.994321614111393,8105,0.9933374460209747,0.9820749893959929,0.9988520591189554,0.9941445301342473,0.9964927349509699,0.9765395894428153,0.9794117647058823,0.9779735682819384,0.9522012578616352,0.9921363040629095,0.9717586649550706,6961,7,34,3,333,4,5,1,757,1103,336,4,1,762,48.0,1
+1,7500,0,15284,0.0344943289855362,0.9923449358806595,0.9961113136468587,0.9897367785559044,0.9929138150324026,0.9997605300062264,0.9997960071097887,6970,32,85,8197,0.5,0.3389629125595093,0.9943884631629759,0.9929138150324026,8105,0.9942011104256632,0.9826996086228116,0.9985677456316242,0.9957155098543273,0.9971395881006865,0.982089552238806,0.9676470588235294,0.9748148148148148,0.9606598984771574,0.9921363040629095,0.9761444229529336,6972,5,25,5,329,6,5,1,757,1103,333,7,1,762,48.0,0
+1,8750,0,15284,0.030296484990695233,0.9947003402250719,0.9954089646007007,0.9948080173871046,0.9951084002657166,0.9998186946414248,0.9998464119156345,6964,38,43,8239,0.5,0.42074909806251526,0.9952995058454863,0.9951084002657166,8105,0.994694632942628,0.9868751212108838,0.9988533753762362,0.99528706083976,0.9970670291151013,0.9853372434017595,0.9882352941176471,0.986784140969163,0.9618805590851334,0.9921363040629095,0.9767741935483871,6969,4,29,3,336,1,5,1,757,1103,339,1,1,762,48.0,1
+1,10000,0,15284,0.029806744011869234,0.9947003402250719,0.9955287009063444,0.9946872736054093,0.9951078093857583,0.9998384047446064,0.9998649862026155,6965,37,44,8238,0.5,0.41207534074783325,0.9954172696575011,0.9951078093857583,8105,0.994694632942628,0.9861554823667197,0.9991396616002294,0.995144244501571,0.9971379507727532,0.9795918367346939,0.9882352941176471,0.9838945827232797,0.9619289340101523,0.9934469200524246,0.9774339136041263,6968,6,28,2,336,2,4,1,758,1103,338,2,1,762,48.1,0
+1,10018,1,15284,0.029792375548663985,0.9947003402250719,0.9955287009063444,0.9946872736054093,0.9951078093857583,0.9998384392329759,0.9998650167494582,6965,37,44,8238,0.5,0.41207534074783325,0.9954172696575011,0.9951078093857583,8105,0.994694632942628,0.9861554823667197,0.9991396616002294,0.995144244501571,0.9971379507727532,0.9795918367346939,0.9882352941176471,0.9838945827232797,0.9619289340101523,0.9934469200524246,0.9774339136041263,6968,6,28,2,336,2,4,1,758,1103,338,2,1,762,47.8,0
diff --git a/presentation/checkpoints/finetune/ft-focal/metrics.csv b/presentation/checkpoints/finetune/ft-focal/metrics.csv
new file mode 100644
index 0000000000000000000000000000000000000000..7008103e279bd8c53a3773561dc6940e3ee7f5ba
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-focal/metrics.csv
@@ -0,0 +1,507 @@
+step,epoch,loss,lr,examples_per_s,max_alloc_gb,max_reserved_gb,elapsed_s,eta_s,val_loss,val_f1,val_roc_auc,val_macro_f1_3way
+20,0,0.31572,1.0465116279069768e-06,59.3,12.957,13.131,10.8,5393,,,,
+40,0,0.20547,2.043189368770764e-06,63.2,12.957,13.131,20.9,5216,,,,
+60,0,0.16318,3.0398671096345517e-06,81.8,12.957,13.131,28.7,4770,,,,
+80,0,0.10407,4.036544850498338e-06,91.7,12.957,13.131,35.7,4437,,,,
+100,0,0.08591,5.033222591362126e-06,80.4,12.957,13.131,43.7,4332,,,,
+120,0,0.06636,6.029900332225914e-06,85.3,12.957,13.131,51.2,4222,,,,
+140,0,0.06927,7.026578073089702e-06,72.6,12.957,13.131,60.0,4233,,,,
+160,0,0.05596,8.023255813953488e-06,85.2,12.957,13.131,67.5,4159,,,,
+180,0,0.05257,9.019933554817277e-06,66.1,12.957,13.131,77.2,4219,,,,
+200,0,0.0445,1.0016611295681064e-05,103.7,12.957,13.131,83.4,4092,,,,
+220,0,0.06655,1.1013289036544852e-05,62.6,12.957,13.131,93.6,4168,,,,
+240,0,0.0438,1.2009966777408639e-05,81.8,12.957,13.131,101.4,4131,,,,
+260,0,0.0723,1.3006644518272425e-05,69.6,12.957,13.131,110.6,4151,,,,
+280,0,0.03788,1.4003322259136214e-05,67.1,12.957,13.131,120.1,4178,,,,
+300,0,0.05209,1.5e-05,69.7,12.957,13.131,129.3,4189,,,,
+320,0,0.04422,1.5996677740863787e-05,65.8,12.957,13.131,139.0,4214,,,,
+340,0,0.06758,1.6993355481727575e-05,64.8,12.957,13.131,148.9,4239,,,,
+360,0,0.02432,1.7990033222591364e-05,95.1,12.957,13.131,155.7,4176,,,,
+380,0,0.04247,1.898671096345515e-05,80.4,12.957,13.131,163.6,4150,,,,
+400,0,0.02887,1.9983388704318937e-05,69.3,12.957,13.131,172.9,4156,,,,
+420,0,0.03664,2.0980066445182726e-05,87.2,12.957,13.131,180.2,4118,,,,
+440,0,0.03521,2.197674418604651e-05,61.8,12.957,13.131,190.5,4148,,,,
+460,0,0.01691,2.29734219269103e-05,95.8,12.957,13.131,197.2,4098,,,,
+480,0,0.04273,2.3970099667774087e-05,70.2,12.957,13.131,206.3,4100,,,,
+500,0,0.03774,2.4966777408637872e-05,66.2,12.957,13.131,216.0,4112,,,,
+520,0,0.04185,2.596345514950166e-05,73.0,12.957,13.133,224.8,4105,,,,
+540,0,0.02567,2.696013289036545e-05,79.8,12.957,13.133,232.8,4086,,,,
+560,0,0.02496,2.7956810631229237e-05,71.3,12.957,13.133,241.8,4083,,,,
+580,0,0.05293,2.8953488372093026e-05,67.7,12.957,13.133,251.2,4088,,,,
+600,0,0.02553,2.9950166112956814e-05,75.2,12.957,13.133,259.7,4077,,,,
+620,0,0.04669,2.9942650807136788e-05,90.4,12.957,13.133,266.8,4044,,,,
+640,0,0.0489,2.9878929481733223e-05,92.0,12.957,13.133,273.7,4011,,,,
+660,0,0.0295,2.981520815632965e-05,53.4,12.957,13.133,285.7,4051,,,,
+680,0,0.03998,2.9751486830926084e-05,71.8,12.957,13.133,294.6,4046,,,,
+700,0,0.03039,2.9687765505522515e-05,75.3,12.957,13.133,303.1,4035,,,,
+720,0,0.05099,2.9624044180118947e-05,56.7,12.957,13.133,314.0,4055,,,,
+740,0,0.02071,2.956032285471538e-05,78.3,12.957,13.133,322.2,4039,,,,
+760,0,0.03734,2.949660152931181e-05,80.5,12.957,13.133,330.1,4021,,,,
+780,0,0.03011,2.9432880203908243e-05,83.7,12.957,13.133,337.8,4000,,,,
+800,0,0.025,2.936915887850467e-05,71.1,12.957,13.133,346.8,3996,,,,
+820,0,0.02726,2.9305437553101107e-05,66.7,12.957,13.133,356.4,3997,,,,
+840,0,0.01782,2.924171622769754e-05,65.1,12.957,13.133,366.2,4001,,,,
+860,0,0.01715,2.9177994902293967e-05,90.2,12.957,13.133,373.3,3975,,,,
+880,0,0.02061,2.91142735768904e-05,70.3,12.957,13.133,382.4,3971,,,,
+900,0,0.02372,2.905055225148683e-05,72.5,12.957,13.133,391.2,3963,,,,
+920,0,0.01724,2.8986830926083263e-05,96.8,12.957,13.133,397.8,3934,,,,
+940,0,0.04256,2.8923109600679695e-05,70.9,12.957,13.133,406.8,3929,,,,
+960,0,0.00629,2.8859388275276127e-05,79.6,12.957,13.133,414.9,3915,,,,
+980,0,0.05258,2.8795666949872555e-05,79.2,12.957,13.133,423.0,3901,,,,
+1000,0,0.04507,2.873194562446899e-05,65.3,12.957,13.133,432.8,3903,,,,
+1020,0,0.02685,2.8668224299065423e-05,70.7,12.957,13.133,441.8,3898,,,,
+1040,0,0.0375,2.860450297366185e-05,67.0,12.957,13.133,451.4,3897,,,,
+1060,0,0.02456,2.8540781648258283e-05,73.6,12.957,13.133,460.1,3888,,,,
+1080,0,0.02632,2.8477060322854715e-05,62.8,12.957,13.133,470.3,3892,,,,
+1100,0,0.02194,2.841333899745115e-05,70.0,12.957,13.133,479.4,3887,,,,
+1120,0,0.01823,2.834961767204758e-05,71.8,12.957,13.133,488.3,3879,,,,
+1140,0,0.02935,2.828589634664401e-05,74.4,12.957,13.133,496.9,3870,,,,
+1160,0,0.02407,2.8222175021240443e-05,82.7,12.957,13.133,504.6,3854,,,,
+1180,0,0.01289,2.8158453695836875e-05,85.1,12.957,13.133,512.2,3836,,,,
+1200,0,0.02414,2.8094732370433307e-05,76.2,12.957,13.133,520.6,3825,,,,
+1220,0,0.00935,2.803101104502974e-05,83.6,12.957,13.133,528.2,3809,,,,
+1240,0,0.02094,2.7967289719626167e-05,61.0,12.957,13.133,538.7,3814,,,,
+1250,0,0.02604,2.7935429056924385e-05,93.4,12.957,13.133,542.1,3803,0.11709,0.98517,0.99829,0.94968
+1260,0,0.00906,2.79035683942226e-05,95.1,12.957,13.133,600.3,3792,,,,
+1280,0,0.02537,2.7839847068819034e-05,62.4,12.957,13.133,610.6,3794,,,,
+1300,0,0.03856,2.7776125743415463e-05,73.3,12.957,13.133,619.3,3785,,,,
+1320,0,0.02362,2.7712404418011895e-05,67.4,12.957,13.133,628.8,3782,,,,
+1340,0,0.01384,2.7648683092608327e-05,78.6,12.957,13.133,636.9,3770,,,,
+1360,0,0.02172,2.758496176720476e-05,66.7,12.957,13.133,646.5,3767,,,,
+1380,0,0.01534,2.752124044180119e-05,67.8,12.957,13.133,656.0,3763,,,,
+1400,0,0.01824,2.7457519116397622e-05,64.8,12.957,13.133,665.8,3761,,,,
+1420,0,0.01713,2.739379779099405e-05,70.5,12.957,13.133,674.9,3755,,,,
+1440,0,0.01775,2.7330076465590486e-05,67.0,12.957,13.133,684.5,3751,,,,
+1460,0,0.02523,2.7266355140186918e-05,62.4,12.957,13.133,694.7,3751,,,,
+1480,0,0.01327,2.720263381478335e-05,66.8,12.957,13.133,704.3,3747,,,,
+1500,0,0.0126,2.713891248937978e-05,75.2,12.957,13.133,712.8,3737,,,,
+1520,0,0.02458,2.707519116397621e-05,74.2,12.957,13.133,721.4,3727,,,,
+1540,0,0.01716,2.7011469838572646e-05,74.3,12.957,13.133,730.0,3717,,,,
+1560,0,0.01699,2.6947748513169074e-05,81.0,12.957,13.133,737.9,3704,,,,
+1580,0,0.01198,2.6884027187765506e-05,63.2,12.957,13.133,748.1,3702,,,,
+1600,0,0.0161,2.6820305862361938e-05,73.5,12.957,13.133,756.8,3693,,,,
+1620,0,0.02587,2.675658453695837e-05,63.2,12.957,13.133,766.9,3691,,,,
+1640,0,0.02691,2.6692863211554802e-05,70.2,12.957,13.133,776.0,3684,,,,
+1660,0,0.02027,2.6629141886151234e-05,75.9,12.957,13.133,784.4,3674,,,,
+1680,0,0.01661,2.6565420560747662e-05,62.3,12.957,13.133,794.7,3672,,,,
+1700,0,0.02096,2.6501699235344094e-05,58.3,12.957,13.133,805.7,3674,,,,
+1720,0,0.0145,2.643797790994053e-05,71.9,12.957,13.133,814.6,3665,,,,
+1740,0,0.01958,2.6374256584536958e-05,69.3,12.957,13.133,823.8,3658,,,,
+1760,0,0.01531,2.631053525913339e-05,66.4,12.957,13.133,833.4,3653,,,,
+1780,0,0.0093,2.6246813933729822e-05,76.4,12.957,13.133,841.8,3642,,,,
+1800,0,0.01629,2.6183092608326254e-05,71.4,12.957,13.133,850.8,3634,,,,
+1820,0,0.01347,2.6119371282922686e-05,72.7,12.957,13.133,859.6,3625,,,,
+1840,0,0.01559,2.6055649957519118e-05,94.8,12.957,13.133,866.3,3607,,,,
+1860,0,0.01413,2.599192863211555e-05,78.2,12.957,13.133,874.5,3595,,,,
+1880,0,0.01487,2.5928207306711978e-05,57.9,12.957,13.133,885.6,3596,,,,
+1900,0,0.01847,2.5864485981308413e-05,76.2,12.957,13.133,894.0,3585,,,,
+1920,0,0.02177,2.5800764655904845e-05,67.5,12.957,13.133,903.4,3579,,,,
+1940,0,0.00836,2.5737043330501274e-05,81.1,12.957,13.133,911.3,3567,,,,
+1960,0,0.0141,2.5673322005097706e-05,89.5,12.957,13.133,918.5,3551,,,,
+1980,0,0.02627,2.560960067969414e-05,66.8,12.957,13.133,928.1,3545,,,,
+2000,0,0.01714,2.554587935429057e-05,70.6,12.957,13.133,937.1,3537,,,,
+2020,0,0.01558,2.5482158028887e-05,65.8,12.957,13.133,946.9,3532,,,,
+2040,0,0.02633,2.5418436703483433e-05,71.5,12.957,13.133,955.8,3524,,,,
+2060,0,0.02136,2.5354715378079862e-05,62.4,12.957,13.133,966.1,3520,,,,
+2080,0,0.02198,2.5290994052676297e-05,62.3,12.957,13.133,976.3,3517,,,,
+2100,0,0.00942,2.522727272727273e-05,67.1,12.957,13.133,985.9,3511,,,,
+2120,0,0.01823,2.5163551401869158e-05,62.5,12.957,13.133,996.1,3507,,,,
+2140,0,0.02082,2.509983007646559e-05,53.9,12.957,13.133,1008.0,3509,,,,
+2160,0,0.00954,2.5036108751062025e-05,68.2,12.957,13.133,1017.4,3502,,,,
+2180,0,0.01296,2.4972387425658453e-05,73.4,12.957,13.133,1026.1,3492,,,,
+2200,0,0.01557,2.4908666100254885e-05,74.9,12.957,13.133,1034.6,3482,,,,
+2220,0,0.01395,2.4844944774851317e-05,60.4,12.957,13.133,1045.2,3479,,,,
+2240,0,0.01257,2.478122344944775e-05,79.5,12.957,13.133,1053.3,3467,,,,
+2260,0,0.0187,2.471750212404418e-05,63.5,12.957,13.133,1063.4,3462,,,,
+2280,0,0.02418,2.4653780798640613e-05,61.2,12.957,13.133,1073.8,3458,,,,
+2300,0,0.01194,2.4590059473237045e-05,69.3,12.957,13.133,1083.1,3450,,,,
+2320,0,0.01199,2.4526338147833473e-05,72.4,12.957,13.133,1091.9,3441,,,,
+2340,0,0.00899,2.446261682242991e-05,81.8,12.957,13.133,1099.7,3429,,,,
+2360,0,0.0116,2.439889549702634e-05,62.0,12.957,13.133,1110.0,3424,,,,
+2380,0,0.00673,2.433517417162277e-05,73.8,12.957,13.133,1118.7,3414,,,,
+2400,0,0.01106,2.42714528462192e-05,77.0,12.957,13.133,1127.0,3403,,,,
+2420,0,0.02458,2.4207731520815636e-05,82.3,12.957,13.133,1134.8,3391,,,,
+2440,0,0.0077,2.4144010195412065e-05,81.3,12.957,13.133,1142.7,3379,,,,
+2460,0,0.03085,2.4080288870008497e-05,62.9,12.957,13.133,1152.9,3374,,,,
+2480,0,0.01485,2.401656754460493e-05,63.0,12.957,13.133,1163.0,3368,,,,
+2500,0,0.00771,2.3952846219201357e-05,66.3,12.957,13.133,1172.7,3362,0.04746,0.98977,0.99922,0.96169
+2520,0,0.02056,2.3889124893797792e-05,69.5,12.957,13.133,1236.0,3354,,,,
+2540,0,0.01698,2.3825403568394224e-05,64.1,12.957,13.133,1245.9,3348,,,,
+2560,0,0.0187,2.3761682242990653e-05,65.6,12.957,13.133,1255.7,3341,,,,
+2580,0,0.01506,2.3697960917587085e-05,82.9,12.957,13.133,1263.4,3328,,,,
+2600,0,0.01128,2.363423959218352e-05,67.8,12.957,13.133,1272.9,3321,,,,
+2620,0,0.01661,2.3570518266779952e-05,64.9,12.957,13.133,1282.7,3315,,,,
+2640,0,0.01092,2.350679694137638e-05,77.9,12.957,13.133,1290.9,3303,,,,
+2660,0,0.0134,2.3443075615972812e-05,88.3,12.957,13.133,1298.2,3290,,,,
+2680,0,0.00951,2.3379354290569244e-05,73.1,12.957,13.133,1306.9,3280,,,,
+2700,0,0.01514,2.3315632965165676e-05,64.2,12.957,13.133,1316.9,3274,,,,
+2720,0,0.01517,2.3251911639762108e-05,78.3,12.957,13.133,1325.1,3263,,,,
+2740,0,0.01236,2.318819031435854e-05,65.8,12.957,13.133,1334.8,3256,,,,
+2760,0,0.01323,2.312446898895497e-05,69.1,12.957,13.133,1344.1,3248,,,,
+2780,0,0.01305,2.3060747663551404e-05,72.3,12.957,13.133,1352.9,3239,,,,
+2800,0,0.01603,2.2997026338147836e-05,60.2,12.957,13.133,1363.6,3234,,,,
+2820,0,0.00917,2.2933305012744264e-05,85.2,12.957,13.133,1371.1,3222,,,,
+2840,0,0.02412,2.2869583687340696e-05,59.3,12.957,13.133,1381.9,3217,,,,
+2860,0,0.00972,2.2805862361937128e-05,61.2,12.957,13.133,1392.3,3212,,,,
+2880,0,0.00561,2.274214103653356e-05,74.9,12.957,13.133,1400.8,3202,,,,
+2900,0,0.01024,2.2678419711129992e-05,77.9,12.957,13.133,1409.1,3191,,,,
+2920,0,0.01401,2.2614698385726424e-05,78.3,12.957,13.133,1417.2,3180,,,,
+2940,0,0.01873,2.2550977060322852e-05,61.0,12.957,13.133,1427.7,3175,,,,
+2960,0,0.01172,2.2487255734919288e-05,77.2,12.957,13.133,1436.0,3164,,,,
+2980,0,0.00484,2.242353440951572e-05,92.6,12.957,13.133,1442.9,3151,,,,
+3000,0,0.00649,2.235981308411215e-05,66.9,12.957,13.133,1452.5,3143,,,,
+3020,0,0.00974,2.229609175870858e-05,87.2,12.957,13.133,1459.8,3130,,,,
+3040,0,0.01907,2.2232370433305012e-05,76.1,12.957,13.133,1468.2,3120,,,,
+3060,0,0.00822,2.2168649107901447e-05,80.2,12.957,13.133,1476.2,3109,,,,
+3080,0,0.01883,2.2104927782497876e-05,67.0,12.957,13.133,1485.8,3102,,,,
+3100,0,0.01287,2.2041206457094308e-05,65.5,12.957,13.133,1495.5,3094,,,,
+3120,0,0.01047,2.197748513169074e-05,70.4,12.957,13.133,1504.6,3086,,,,
+3140,0,0.01787,2.191376380628717e-05,69.2,12.957,13.133,1513.9,3078,,,,
+3160,0,0.00862,2.1850042480883604e-05,81.3,12.957,13.133,1521.7,3066,,,,
+3180,0,0.01055,2.1786321155480035e-05,70.3,12.957,13.133,1530.8,3058,,,,
+3200,0,0.01365,2.1722599830076464e-05,68.3,12.957,13.133,1540.2,3050,,,,
+3220,0,0.01119,2.16588785046729e-05,73.2,12.957,13.133,1549.0,3040,,,,
+3240,0,0.00497,2.159515717926933e-05,67.8,12.957,13.133,1558.4,3032,,,,
+3260,0,0.01991,2.153143585386576e-05,69.2,12.957,13.133,1567.6,3024,,,,
+3280,0,0.00639,2.146771452846219e-05,65.4,12.957,13.133,1577.4,3017,,,,
+3300,0,0.02166,2.1403993203058624e-05,85.5,12.957,13.133,1584.9,3005,,,,
+3320,0,0.01347,2.1340271877655055e-05,73.5,12.957,13.133,1593.6,2995,,,,
+3340,0,0.01185,2.1276550552251487e-05,100.8,12.957,13.133,1600.0,2981,,,,
+3360,0,0.00631,2.121282922684792e-05,66.4,12.957,13.133,1609.6,2974,,,,
+3380,0,0.01726,2.1149107901444348e-05,67.1,12.957,13.133,1619.2,2966,,,,
+3400,0,0.01705,2.1085386576040783e-05,79.5,12.957,13.133,1627.2,2955,,,,
+3420,0,0.01836,2.1021665250637215e-05,72.5,12.957,13.133,1636.0,2946,,,,
+3440,0,0.01818,2.0957943925233647e-05,68.3,12.957,13.133,1645.4,2938,,,,
+3460,0,0.00968,2.0894222599830075e-05,73.0,12.957,13.133,1654.2,2929,,,,
+3480,0,0.01378,2.0830501274426507e-05,57.3,12.957,13.133,1665.3,2924,,,,
+3500,0,0.00892,2.0766779949022943e-05,84.6,12.957,13.133,1672.9,2913,,,,
+3520,0,0.00682,2.070305862361937e-05,94.8,12.957,13.133,1679.7,2900,,,,
+3540,0,0.00822,2.0639337298215803e-05,76.4,12.957,13.133,1688.0,2890,,,,
+3560,0,0.00762,2.0575615972812235e-05,79.7,12.957,13.133,1696.1,2879,,,,
+3580,0,0.00878,2.0511894647408667e-05,74.5,12.957,13.133,1704.7,2870,,,,
+3600,0,0.0091,2.04481733220051e-05,66.9,12.957,13.133,1714.2,2862,,,,
+3620,0,0.01224,2.038445199660153e-05,69.1,12.957,13.133,1723.5,2854,,,,
+3640,0,0.01045,2.032073067119796e-05,94.9,12.957,13.133,1730.2,2841,,,,
+3660,0,0.00638,2.025700934579439e-05,75.2,12.957,13.133,1738.7,2831,,,,
+3680,0,0.01101,2.0193288020390827e-05,69.1,12.957,13.133,1748.0,2823,,,,
+3700,0,0.00439,2.0129566694987255e-05,85.4,12.957,13.133,1755.5,2812,,,,
+3720,0,0.00502,2.0065845369583687e-05,69.0,12.957,13.133,1764.8,2803,,,,
+3740,0,0.01368,2.000212404418012e-05,67.6,12.957,13.133,1774.2,2795,,,,
+3750,0,0.01792,1.9970263381478337e-05,75.3,12.957,13.133,1778.5,2791,0.0566,0.99084,0.99956,0.98357
+3760,0,0.00745,1.993840271877655e-05,72.7,12.957,13.133,1836.3,2786,,,,
+3780,0,0.01229,1.9874681393372983e-05,76.6,12.957,13.133,1844.7,2776,,,,
+3800,0,0.01534,1.9810960067969415e-05,73.5,12.957,13.133,1853.4,2767,,,,
+3820,0,0.00897,1.9747238742565847e-05,70.4,12.957,13.133,1862.5,2759,,,,
+3840,0,0.01721,1.9683517417162275e-05,78.8,12.957,13.133,1870.6,2748,,,,
+3860,0,0.00496,1.961979609175871e-05,68.1,12.957,13.133,1880.0,2740,,,,
+3880,0,0.01297,1.9556074766355142e-05,75.7,12.957,13.133,1888.4,2731,,,,
+3900,0,0.01119,1.949235344095157e-05,71.5,12.957,13.133,1897.4,2722,,,,
+3920,0,0.01833,1.9428632115548003e-05,71.6,12.957,13.133,1906.3,2713,,,,
+3940,0,0.01683,1.9364910790144438e-05,61.5,12.957,13.133,1916.7,2706,,,,
+3960,0,0.00663,1.9301189464740867e-05,76.6,12.957,13.133,1925.1,2697,,,,
+3980,0,0.02949,1.92374681393373e-05,83.7,12.957,13.133,1932.7,2686,,,,
+4000,0,0.00667,1.917374681393373e-05,68.8,12.957,13.133,1942.0,2678,,,,
+4020,0,0.00902,1.9110025488530162e-05,71.1,12.957,13.133,1951.0,2669,,,,
+4040,0,0.01515,1.9046304163126594e-05,64.1,12.957,13.133,1961.0,2662,,,,
+4060,0,0.00993,1.8982582837723026e-05,75.3,12.957,13.133,1969.5,2652,,,,
+4080,0,0.01648,1.8918861512319455e-05,58.8,12.957,13.133,1980.4,2646,,,,
+4100,0,0.00692,1.8855140186915887e-05,73.0,12.957,13.133,1989.1,2637,,,,
+4120,0,0.00691,1.8791418861512322e-05,74.2,12.957,13.133,1997.8,2628,,,,
+4140,0,0.01242,1.872769753610875e-05,65.4,12.957,13.133,2007.5,2620,,,,
+4160,0,0.02724,1.8663976210705182e-05,70.2,12.957,13.133,2016.7,2611,,,,
+4180,0,0.01154,1.8600254885301614e-05,64.9,12.957,13.133,2026.5,2604,,,,
+4200,0,0.00723,1.853653355989805e-05,73.5,12.957,13.133,2035.2,2594,,,,
+4220,0,0.00492,1.8472812234494478e-05,95.6,12.957,13.133,2041.9,2583,,,,
+4240,0,0.01103,1.840909090909091e-05,89.7,12.957,13.133,2049.1,2571,,,,
+4260,0,0.00957,1.8345369583687342e-05,58.8,12.957,13.133,2059.9,2565,,,,
+4280,0,0.0078,1.828164825828377e-05,76.1,12.957,13.133,2068.4,2555,,,,
+4300,0,0.01022,1.8217926932880206e-05,59.9,12.957,13.133,2079.0,2549,,,,
+4320,0,0.01665,1.8154205607476638e-05,80.4,12.957,13.133,2087.0,2539,,,,
+4340,0,0.00541,1.8090484282073066e-05,63.0,12.957,13.133,2097.2,2531,,,,
+4360,0,0.0106,1.8026762956669498e-05,61.4,12.957,13.133,2107.6,2524,,,,
+4380,0,0.00599,1.7963041631265933e-05,64.8,12.957,13.133,2117.5,2517,,,,
+4400,0,0.01922,1.7899320305862362e-05,62.1,12.957,13.133,2127.8,2510,,,,
+4420,0,0.01233,1.7835598980458794e-05,68.9,12.957,13.133,2137.1,2501,,,,
+4440,0,0.00879,1.7771877655055226e-05,81.5,12.957,13.133,2144.9,2491,,,,
+4460,0,0.00365,1.7708156329651654e-05,70.7,12.957,13.133,2154.0,2482,,,,
+4480,0,0.00837,1.764443500424809e-05,65.1,12.957,13.133,2163.8,2474,,,,
+4500,0,0.00479,1.758071367884452e-05,73.1,12.957,13.133,2172.6,2465,,,,
+4520,0,0.00896,1.751699235344095e-05,67.3,12.957,13.133,2182.1,2457,,,,
+4540,0,0.00711,1.7453271028037382e-05,77.3,12.957,13.133,2190.4,2447,,,,
+4560,0,0.01282,1.7389549702633817e-05,59.8,12.957,13.133,2201.1,2440,,,,
+4580,0,0.00889,1.732582837723025e-05,62.1,12.957,13.133,2211.4,2433,,,,
+4600,0,0.00859,1.7262107051826678e-05,61.6,12.957,13.133,2221.8,2426,,,,
+4620,0,0.00719,1.719838572642311e-05,66.4,12.957,13.133,2231.4,2418,,,,
+4640,0,0.00932,1.713466440101954e-05,97.0,12.957,13.133,2238.0,2406,,,,
+4660,0,0.01336,1.7070943075615973e-05,63.5,12.957,13.133,2248.1,2398,,,,
+4680,0,0.00739,1.7007221750212405e-05,75.0,12.957,13.133,2256.6,2389,,,,
+4700,0,0.00606,1.6943500424808837e-05,78.9,12.957,13.133,2264.7,2379,,,,
+4720,0,0.0137,1.6879779099405266e-05,62.1,12.957,13.133,2275.0,2371,,,,
+4740,0,0.00496,1.68160577740017e-05,69.6,12.957,13.133,2284.2,2363,,,,
+4760,0,0.01706,1.6752336448598133e-05,69.5,12.957,13.133,2293.4,2354,,,,
+4780,0,0.01064,1.668861512319456e-05,83.4,12.957,13.133,2301.1,2344,,,,
+4800,0,0.00816,1.6624893797790993e-05,71.3,12.957,13.133,2310.1,2335,,,,
+4820,0,0.00727,1.6561172472387425e-05,70.7,12.957,13.133,2319.1,2326,,,,
+4840,0,0.01125,1.6497451146983857e-05,57.1,12.957,13.133,2330.3,2319,,,,
+4860,0,0.005,1.643372982158029e-05,81.9,12.957,13.133,2338.1,2309,,,,
+4880,0,0.0022,1.637000849617672e-05,72.4,12.957,13.133,2347.0,2300,,,,
+4900,0,0.00428,1.630628717077315e-05,61.1,12.957,13.133,2357.4,2293,,,,
+4920,0,0.00451,1.6242565845369585e-05,65.5,12.957,13.133,2367.2,2285,,,,
+4940,0,0.00686,1.6178844519966017e-05,70.8,12.957,13.133,2376.3,2276,,,,
+4960,0,0.01062,1.611512319456245e-05,61.6,12.957,13.133,2386.6,2268,,,,
+4980,0,0.0128,1.6051401869158877e-05,64.0,12.957,13.133,2396.6,2260,,,,
+5000,0,0.01237,1.5987680543755312e-05,70.8,12.957,13.133,2405.7,2251,0.04522,0.9933,0.99958,0.97967
+5009,0,0.02499,1.5959005947323706e-05,97.5,12.957,13.133,2461.8,2246,0.05172,0.99338,0.99959,0.97595
+5020,1,0.0041,1.5923959218351744e-05,80.5,12.957,13.133,2519.1,2241,,,,
+5040,1,0.00942,1.5860237892948173e-05,74.6,12.957,13.133,2527.6,2232,,,,
+5060,1,0.00462,1.5796516567544605e-05,72.6,12.957,13.133,2536.4,2222,,,,
+5080,1,0.00479,1.5732795242141037e-05,62.8,12.957,13.133,2546.6,2215,,,,
+5100,1,0.00129,1.566907391673747e-05,85.7,12.957,13.133,2554.1,2204,,,,
+5120,1,0.0116,1.56053525913339e-05,74.5,12.957,13.133,2562.7,2195,,,,
+5140,1,0.00372,1.5541631265930333e-05,66.5,12.957,13.133,2572.3,2187,,,,
+5160,1,0.00251,1.547790994052676e-05,80.7,12.957,13.133,2580.2,2177,,,,
+5180,1,0.00276,1.5414188615123196e-05,107.3,12.957,13.133,2586.2,2165,,,,
+5200,1,0.00267,1.5350467289719628e-05,67.1,12.957,13.133,2595.8,2156,,,,
+5220,1,0.00218,1.5286745964316057e-05,80.3,12.957,13.133,2603.7,2147,,,,
+5240,1,0.00231,1.5223024638912489e-05,61.6,12.957,13.133,2614.1,2139,,,,
+5260,1,0.00823,1.5159303313508922e-05,74.1,12.957,13.133,2622.7,2130,,,,
+5280,1,0.00463,1.509558198810535e-05,71.8,12.957,13.133,2631.6,2121,,,,
+5300,1,0.00384,1.5031860662701784e-05,60.7,12.957,13.133,2642.2,2113,,,,
+5320,1,0.00135,1.4968139337298216e-05,57.3,12.957,13.133,2653.4,2106,,,,
+5340,1,0.01735,1.4904418011894648e-05,70.5,12.957,13.133,2662.4,2097,,,,
+5360,1,0.00395,1.4840696686491078e-05,71.1,12.957,13.133,2671.4,2088,,,,
+5380,1,0.00382,1.4776975361087512e-05,62.1,12.957,13.133,2681.7,2081,,,,
+5400,1,0.00794,1.4713254035683942e-05,61.5,12.957,13.133,2692.1,2073,,,,
+5420,1,0.00514,1.4649532710280374e-05,66.8,12.957,13.133,2701.7,2064,,,,
+5440,1,0.01167,1.4585811384876806e-05,66.4,12.957,13.133,2711.4,2056,,,,
+5460,1,0.01209,1.4522090059473236e-05,66.9,12.957,13.133,2720.9,2047,,,,
+5480,1,0.00558,1.445836873406967e-05,66.2,12.957,13.133,2730.6,2039,,,,
+5500,1,0.00591,1.43946474086661e-05,72.9,12.957,13.133,2739.4,2030,,,,
+5520,1,0.00404,1.4330926083262532e-05,84.9,12.957,13.133,2746.9,2020,,,,
+5540,1,0.00158,1.4267204757858964e-05,60.2,12.957,13.133,2757.5,2012,,,,
+5560,1,0.00842,1.4203483432455396e-05,79.2,12.957,13.133,2765.6,2002,,,,
+5580,1,0.00318,1.4139762107051826e-05,125.1,12.957,13.133,2770.7,1990,,,,
+5600,1,0.00409,1.407604078164826e-05,82.3,12.957,13.133,2778.5,1980,,,,
+5620,1,0.00221,1.401231945624469e-05,70.7,12.957,13.133,2787.6,1971,,,,
+5640,1,0.00222,1.3948598130841122e-05,73.4,12.957,13.133,2796.3,1962,,,,
+5660,1,0.00668,1.3884876805437554e-05,63.9,12.957,13.133,2806.3,1954,,,,
+5680,1,0.00633,1.3821155480033984e-05,72.7,12.957,13.133,2815.1,1945,,,,
+5700,1,0.01596,1.3757434154630418e-05,67.8,12.957,13.133,2824.6,1936,,,,
+5720,1,0.00506,1.3693712829226848e-05,78.0,12.957,13.133,2832.8,1927,,,,
+5740,1,0.00805,1.362999150382328e-05,87.8,12.957,13.133,2840.0,1917,,,,
+5760,1,0.00085,1.3566270178419712e-05,64.2,12.957,13.133,2850.0,1908,,,,
+5780,1,0.01167,1.3502548853016144e-05,64.6,12.957,13.133,2859.9,1900,,,,
+5800,1,0.00548,1.3438827527612575e-05,78.1,12.957,13.133,2868.1,1891,,,,
+5820,1,0.00211,1.3375106202209006e-05,59.6,12.957,13.133,2878.9,1883,,,,
+5840,1,0.00662,1.3311384876805438e-05,69.8,12.957,13.133,2888.0,1874,,,,
+5860,1,0.00288,1.324766355140187e-05,70.4,12.957,13.133,2897.1,1865,,,,
+5880,1,0.00505,1.3183942225998301e-05,62.3,12.957,13.133,2907.4,1857,,,,
+5900,1,0.00224,1.3120220900594732e-05,70.0,12.957,13.133,2916.5,1848,,,,
+5920,1,0.0033,1.3056499575191165e-05,65.6,12.957,13.133,2926.3,1840,,,,
+5940,1,0.00369,1.2992778249787595e-05,65.2,12.957,13.133,2936.1,1832,,,,
+5960,1,0.00686,1.2929056924384027e-05,63.1,12.957,13.133,2946.2,1823,,,,
+5980,1,0.01081,1.286533559898046e-05,65.5,12.957,13.133,2956.0,1815,,,,
+6000,1,0.00968,1.2801614273576891e-05,61.3,12.957,13.133,2966.5,1807,,,,
+6020,1,0.0045,1.2737892948173323e-05,75.4,12.957,13.133,2975.0,1798,,,,
+6040,1,0.00139,1.2674171622769753e-05,75.5,12.957,13.133,2983.4,1788,,,,
+6060,1,0.00412,1.2610450297366185e-05,62.4,12.957,13.133,2993.7,1780,,,,
+6080,1,0.0062,1.2546728971962617e-05,79.3,12.957,13.133,3001.8,1770,,,,
+6100,1,0.0108,1.2483007646559049e-05,65.1,12.957,13.133,3011.6,1762,,,,
+6120,1,0.00187,1.241928632115548e-05,71.4,12.957,13.133,3020.5,1753,,,,
+6140,1,0.01076,1.2355564995751913e-05,92.4,12.957,13.133,3027.5,1743,,,,
+6160,1,0.00051,1.2291843670348343e-05,65.3,12.957,13.133,3037.3,1734,,,,
+6180,1,0.0015,1.2228122344944777e-05,69.5,12.957,13.133,3046.5,1725,,,,
+6200,1,0.00535,1.2164401019541207e-05,68.9,12.957,13.133,3055.8,1717,,,,
+6220,1,0.00636,1.2100679694137637e-05,77.9,12.957,13.133,3064.0,1707,,,,
+6240,1,0.00718,1.203695836873407e-05,79.8,12.957,13.133,3072.0,1697,,,,
+6250,1,0.00387,1.2005097706032285e-05,47.6,12.957,13.133,3078.7,1694,0.03721,0.99432,0.99977,0.98207
+6260,1,0.00918,1.1973237043330501e-05,73.2,12.957,13.133,3136.3,1690,,,,
+6280,1,0.00401,1.1909515717926933e-05,104.0,12.957,13.133,3142.5,1679,,,,
+6300,1,0.01166,1.1845794392523365e-05,66.6,12.957,13.133,3152.1,1670,,,,
+6320,1,0.0015,1.1782073067119797e-05,62.8,12.957,13.133,3162.2,1662,,,,
+6340,1,0.00113,1.1718351741716227e-05,77.8,12.957,13.133,3170.5,1653,,,,
+6360,1,0.00256,1.165463041631266e-05,112.6,12.957,13.133,3176.2,1642,,,,
+6380,1,0.00352,1.159090909090909e-05,69.6,12.957,13.133,3185.4,1633,,,,
+6400,1,0.0021,1.1527187765505523e-05,87.8,12.957,13.133,3192.7,1623,,,,
+6420,1,0.01109,1.1463466440101955e-05,66.1,12.957,13.133,3202.3,1615,,,,
+6440,1,0.01216,1.1399745114698385e-05,63.4,12.957,13.133,3212.4,1606,,,,
+6460,1,0.00196,1.1336023789294818e-05,73.8,12.957,13.133,3221.1,1597,,,,
+6480,1,0.00139,1.1272302463891249e-05,70.9,12.957,13.133,3230.1,1588,,,,
+6500,1,0.00283,1.120858113848768e-05,86.7,12.957,13.133,3237.5,1578,,,,
+6520,1,0.00365,1.1144859813084113e-05,76.7,12.957,13.133,3245.9,1569,,,,
+6540,1,0.005,1.1081138487680544e-05,72.8,12.957,13.133,3254.7,1560,,,,
+6560,1,0.00812,1.1017417162276975e-05,82.0,12.957,13.133,3262.5,1550,,,,
+6580,1,0.00304,1.0953695836873408e-05,68.4,12.957,13.133,3271.8,1542,,,,
+6600,1,0.00145,1.0889974511469838e-05,71.1,12.957,13.133,3280.8,1533,,,,
+6620,1,0.002,1.082625318606627e-05,58.6,12.957,13.133,3291.7,1525,,,,
+6640,1,0.00221,1.0762531860662702e-05,57.3,12.957,13.133,3302.9,1517,,,,
+6660,1,0.00608,1.0698810535259133e-05,93.1,12.957,13.133,3309.8,1507,,,,
+6680,1,0.00261,1.0635089209855566e-05,74.9,12.957,13.133,3318.3,1498,,,,
+6700,1,0.00223,1.0571367884451996e-05,68.6,12.957,13.133,3327.7,1489,,,,
+6720,1,0.00349,1.0507646559048428e-05,63.8,12.957,13.133,3337.7,1480,,,,
+6740,1,0.00087,1.044392523364486e-05,67.2,12.957,13.133,3347.2,1472,,,,
+6760,1,0.00703,1.0380203908241292e-05,62.7,12.957,13.133,3357.4,1463,,,,
+6780,1,0.00481,1.0316482582837724e-05,64.3,12.957,13.133,3367.4,1455,,,,
+6800,1,0.00271,1.0252761257434154e-05,77.5,12.957,13.133,3375.6,1445,,,,
+6820,1,0.00338,1.0189039932030586e-05,59.7,12.957,13.133,3386.4,1437,,,,
+6840,1,0.00087,1.0125318606627018e-05,85.2,12.957,13.133,3393.9,1427,,,,
+6860,1,0.00793,1.006159728122345e-05,69.8,12.957,13.133,3403.0,1419,,,,
+6880,1,0.00142,9.99787595581988e-06,82.8,12.957,13.133,3410.8,1409,,,,
+6900,1,0.00443,9.934154630416314e-06,82.6,12.957,13.133,3418.5,1399,,,,
+6920,1,0.00635,9.870433305012744e-06,64.5,12.957,13.133,3428.4,1391,,,,
+6940,1,0.00372,9.806711979609176e-06,60.7,12.957,13.133,3439.0,1383,,,,
+6960,1,0.0103,9.742990654205608e-06,85.1,12.957,13.133,3446.5,1373,,,,
+6980,1,0.00115,9.67926932880204e-06,84.5,12.957,13.133,3454.1,1363,,,,
+7000,1,0.00129,9.615548003398472e-06,65.8,12.957,13.133,3463.8,1355,,,,
+7020,1,0.00386,9.551826677994902e-06,83.9,12.957,13.133,3471.4,1345,,,,
+7040,1,0.00115,9.488105352591334e-06,77.8,12.957,13.133,3479.7,1336,,,,
+7060,1,0.0073,9.424384027187766e-06,64.3,12.957,13.133,3489.6,1327,,,,
+7080,1,0.01158,9.360662701784198e-06,73.5,12.957,13.133,3498.3,1318,,,,
+7100,1,0.00125,9.296941376380628e-06,68.7,12.957,13.133,3507.6,1309,,,,
+7120,1,0.00277,9.233220050977061e-06,73.0,12.957,13.133,3516.4,1300,,,,
+7140,1,0.00277,9.169498725573492e-06,67.6,12.957,13.133,3525.9,1292,,,,
+7160,1,0.0008,9.105777400169925e-06,57.5,12.957,13.133,3537.0,1283,,,,
+7180,1,0.00083,9.042056074766356e-06,83.6,12.957,13.133,3544.7,1274,,,,
+7200,1,0.0074,8.978334749362786e-06,57.6,12.957,13.133,3555.8,1266,,,,
+7220,1,0.00139,8.91461342395922e-06,77.5,12.957,13.133,3564.0,1257,,,,
+7240,1,0.00173,8.85089209855565e-06,102.2,12.957,13.133,3570.3,1247,,,,
+7260,1,0.00242,8.787170773152081e-06,56.8,12.957,13.133,3581.6,1238,,,,
+7280,1,0.00345,8.723449447748513e-06,62.8,12.957,13.133,3591.7,1230,,,,
+7300,1,0.00355,8.659728122344945e-06,59.1,12.957,13.133,3602.6,1222,,,,
+7320,1,0.0006,8.596006796941376e-06,87.1,12.957,13.133,3609.9,1212,,,,
+7340,1,0.00146,8.532285471537809e-06,82.6,12.957,13.133,3617.7,1203,,,,
+7360,1,0.00485,8.46856414613424e-06,74.4,12.957,13.133,3626.3,1193,,,,
+7380,1,0.00467,8.404842820730673e-06,83.9,12.957,13.133,3633.9,1184,,,,
+7400,1,0.00068,8.341121495327103e-06,69.4,12.957,13.133,3643.1,1175,,,,
+7420,1,0.0019,8.277400169923533e-06,69.9,12.957,13.133,3652.3,1166,,,,
+7440,1,0.00287,8.213678844519967e-06,65.6,12.957,13.133,3662.0,1158,,,,
+7460,1,0.00365,8.149957519116397e-06,67.6,12.957,13.133,3671.5,1149,,,,
+7480,1,0.00199,8.086236193712829e-06,88.4,12.957,13.133,3678.7,1139,,,,
+7500,1,0.00733,8.022514868309261e-06,75.2,12.957,13.133,3687.3,1130,0.03449,0.99291,0.99976,0.9827
+7520,1,0.00402,7.958793542905693e-06,61.1,12.957,13.133,3749.8,1122,,,,
+7540,1,0.00454,7.895072217502125e-06,69.2,12.957,13.133,3759.1,1113,,,,
+7560,1,0.00428,7.831350892098557e-06,74.5,12.957,13.133,3767.6,1104,,,,
+7580,1,0.00437,7.767629566694987e-06,60.8,12.957,13.133,3778.2,1095,,,,
+7600,1,0.00342,7.703908241291419e-06,77.3,12.957,13.133,3786.4,1086,,,,
+7620,1,0.00113,7.64018691588785e-06,89.8,12.957,13.133,3793.6,1076,,,,
+7640,1,0.00261,7.576465590484282e-06,78.6,12.957,13.133,3801.7,1067,,,,
+7660,1,0.0035,7.512744265080715e-06,64.2,12.957,13.133,3811.7,1058,,,,
+7680,1,0.00104,7.449022939677146e-06,56.2,12.957,13.133,3823.1,1050,,,,
+7700,1,0.00046,7.385301614273577e-06,66.1,12.957,13.133,3832.8,1041,,,,
+7720,1,0.00707,7.321580288870009e-06,68.7,12.957,13.133,3842.1,1032,,,,
+7740,1,0.00766,7.25785896346644e-06,68.7,12.957,13.133,3851.4,1024,,,,
+7760,1,0.00701,7.194137638062872e-06,87.8,12.957,13.133,3858.7,1014,,,,
+7780,1,0.00398,7.130416312659304e-06,72.3,12.957,13.133,3867.5,1005,,,,
+7800,1,0.00816,7.0666949872557355e-06,70.0,12.957,13.133,3876.7,996,,,,
+7820,1,0.00406,7.0029736618521666e-06,69.8,12.957,13.133,3885.9,987,,,,
+7840,1,0.00244,6.9392523364485985e-06,76.0,12.957,13.133,3894.3,978,,,,
+7860,1,0.00086,6.87553101104503e-06,72.2,12.957,13.133,3903.1,969,,,,
+7880,1,0.01012,6.811809685641462e-06,63.8,12.957,13.133,3913.2,960,,,,
+7900,1,0.00647,6.7480883602378925e-06,64.7,12.957,13.133,3923.1,952,,,,
+7920,1,0.0068,6.6843670348343244e-06,60.3,12.957,13.133,3933.7,943,,,,
+7940,1,0.00087,6.620645709430756e-06,68.3,12.957,13.133,3943.0,934,,,,
+7960,1,0.00338,6.5569243840271874e-06,92.5,12.957,13.133,3950.0,925,,,,
+7980,1,0.00574,6.493203058623619e-06,68.0,12.957,13.133,3959.4,916,,,,
+8000,1,0.00402,6.429481733220051e-06,74.2,12.957,13.133,3968.0,907,,,,
+8020,1,0.01167,6.365760407816483e-06,67.9,12.957,13.133,3977.4,898,,,,
+8040,1,0.00217,6.302039082412914e-06,69.4,12.957,13.133,3986.6,889,,,,
+8060,1,0.0035,6.238317757009346e-06,68.3,12.957,13.133,3996.0,880,,,,
+8080,1,0.00226,6.174596431605777e-06,65.5,12.957,13.133,4005.8,871,,,,
+8100,1,0.00501,6.110875106202209e-06,57.9,12.957,13.133,4016.8,863,,,,
+8120,1,0.00868,6.04715378079864e-06,63.5,12.957,13.133,4026.9,854,,,,
+8140,1,0.00558,5.983432455395072e-06,68.7,12.957,13.133,4036.2,845,,,,
+8160,1,0.00131,5.919711129991504e-06,60.4,12.957,13.133,4046.8,836,,,,
+8180,1,0.0025,5.855989804587936e-06,57.2,12.957,13.133,4058.0,828,,,,
+8200,1,0.00626,5.792268479184367e-06,58.0,12.957,13.133,4069.0,819,,,,
+8220,1,0.00114,5.728547153780799e-06,65.9,12.957,13.133,4078.8,810,,,,
+8240,1,0.00581,5.664825828377231e-06,63.2,12.957,13.133,4088.9,802,,,,
+8260,1,0.00372,5.601104502973663e-06,85.7,12.957,13.133,4096.3,792,,,,
+8280,1,0.00049,5.537383177570093e-06,91.5,12.957,13.133,4103.3,783,,,,
+8300,1,0.00203,5.473661852166525e-06,79.2,12.957,13.133,4111.4,774,,,,
+8320,1,0.00772,5.409940526762957e-06,67.6,12.957,13.133,4120.9,765,,,,
+8340,1,0.00424,5.346219201359388e-06,104.9,12.957,13.133,4127.0,755,,,,
+8360,1,0.00174,5.28249787595582e-06,62.3,12.957,13.133,4137.2,746,,,,
+8380,1,0.00083,5.218776550552252e-06,72.9,12.957,13.133,4146.0,737,,,,
+8400,1,0.00195,5.155055225148684e-06,72.7,12.957,13.133,4154.8,728,,,,
+8420,1,0.00185,5.091333899745115e-06,84.8,12.957,13.133,4162.4,719,,,,
+8440,1,0.01127,5.027612574341547e-06,74.2,12.957,13.133,4171.0,710,,,,
+8460,1,0.00223,4.9638912489379785e-06,90.8,12.957,13.133,4178.0,701,,,,
+8480,1,0.00592,4.9001699235344096e-06,62.3,12.957,13.133,4188.3,692,,,,
+8500,1,0.01131,4.836448598130841e-06,81.8,12.957,13.133,4196.1,683,,,,
+8520,1,0.00295,4.7727272727272725e-06,75.9,12.957,13.133,4204.6,674,,,,
+8540,1,0.00153,4.7090059473237045e-06,60.2,12.957,13.133,4215.2,665,,,,
+8560,1,0.00222,4.645284621920136e-06,71.6,12.957,13.133,4224.1,656,,,,
+8580,1,0.00105,4.5815632965165674e-06,61.0,12.957,13.133,4234.6,647,,,,
+8600,1,0.01045,4.517841971112999e-06,76.1,12.957,13.133,4243.1,638,,,,
+8620,1,0.00392,4.454120645709431e-06,68.9,12.957,13.133,4252.3,629,,,,
+8640,1,0.00459,4.390399320305863e-06,78.6,12.957,13.133,4260.5,620,,,,
+8660,1,0.00479,4.326677994902294e-06,64.4,12.957,13.133,4270.4,611,,,,
+8680,1,0.00426,4.262956669498725e-06,53.2,12.957,13.133,4282.5,603,,,,
+8700,1,0.00232,4.199235344095157e-06,74.2,12.957,13.133,4291.1,593,,,,
+8720,1,0.00131,4.135514018691588e-06,71.0,12.957,13.133,4300.1,584,,,,
+8740,1,0.00197,4.07179269328802e-06,78.9,12.957,13.133,4308.2,575,,,,
+8750,1,0.00162,4.039932030586237e-06,85.5,12.957,13.133,4312.0,571,0.0303,0.99511,0.99982,0.98688
+8760,1,0.00266,4.008071367884452e-06,65.2,12.957,13.133,4370.1,566,,,,
+8780,1,0.00491,3.944350042480884e-06,88.5,12.957,13.133,4377.3,557,,,,
+8800,1,0.00586,3.880628717077315e-06,68.5,12.957,13.133,4386.7,548,,,,
+8820,1,0.00161,3.816907391673747e-06,100.7,12.957,13.133,4393.0,539,,,,
+8840,1,0.00218,3.7531860662701785e-06,65.7,12.957,13.133,4402.8,530,,,,
+8860,1,0.00417,3.68946474086661e-06,82.6,12.957,13.133,4410.5,521,,,,
+8880,1,0.00214,3.625743415463042e-06,65.3,12.957,13.133,4420.3,512,,,,
+8900,1,0.00069,3.5620220900594734e-06,82.0,12.957,13.133,4428.1,503,,,,
+8920,1,0.00381,3.498300764655905e-06,71.6,12.957,13.133,4437.1,494,,,,
+8940,1,0.00059,3.4345794392523364e-06,63.0,12.957,13.133,4447.2,485,,,,
+8960,1,0.0006,3.3708581138487683e-06,86.9,12.957,13.133,4454.6,476,,,,
+8980,1,0.00366,3.3071367884451998e-06,62.3,12.957,13.133,4464.9,467,,,,
+9000,1,0.00986,3.2434154630416313e-06,71.1,12.957,13.133,4473.9,458,,,,
+9020,1,0.00429,3.1796941376380628e-06,77.5,12.957,13.133,4482.1,449,,,,
+9040,1,0.00092,3.1159728122344942e-06,66.6,12.957,13.133,4491.7,440,,,,
+9060,1,0.00297,3.052251486830926e-06,69.7,12.957,13.133,4500.9,431,,,,
+9080,1,0.00487,2.9885301614273577e-06,61.4,12.957,13.133,4511.3,422,,,,
+9100,1,0.01326,2.9248088360237896e-06,67.9,12.957,13.133,4520.8,413,,,,
+9120,1,0.00185,2.8610875106202206e-06,68.6,12.957,13.133,4530.1,404,,,,
+9140,1,0.00244,2.7973661852166525e-06,67.7,12.957,13.133,4539.5,395,,,,
+9160,1,0.00045,2.733644859813084e-06,61.0,12.957,13.133,4550.0,386,,,,
+9180,1,0.00701,2.669923534409516e-06,94.1,12.957,13.133,4556.8,377,,,,
+9200,1,0.0022,2.6062022090059474e-06,66.5,12.957,13.133,4566.5,368,,,,
+9220,1,0.00313,2.542480883602379e-06,57.8,12.957,13.133,4577.5,359,,,,
+9240,1,0.0025,2.4787595581988104e-06,72.1,12.957,13.133,4586.4,350,,,,
+9260,1,0.00152,2.4150382327952423e-06,65.3,12.957,13.133,4596.2,341,,,,
+9280,1,0.00115,2.351316907391674e-06,89.5,12.957,13.133,4603.4,332,,,,
+9300,1,0.00321,2.2875955819881057e-06,83.2,12.957,13.133,4611.0,323,,,,
+9320,1,0.00675,2.223874256584537e-06,66.7,12.957,13.133,4620.6,314,,,,
+9340,1,0.00254,2.1601529311809687e-06,67.2,12.957,13.133,4630.2,305,,,,
+9360,1,0.00343,2.0964316057774e-06,73.8,12.957,13.133,4638.8,296,,,,
+9380,1,0.00266,2.0327102803738317e-06,73.0,12.957,13.133,4647.6,287,,,,
+9400,1,0.00062,1.9689889549702636e-06,76.3,12.957,13.133,4656.0,278,,,,
+9420,1,0.00054,1.9052676295666949e-06,72.8,12.957,13.133,4664.8,269,,,,
+9440,1,0.0042,1.8415463041631266e-06,74.7,12.957,13.133,4673.3,260,,,,
+9460,1,0.00045,1.7778249787595583e-06,76.8,12.957,13.133,4681.7,251,,,,
+9480,1,0.00227,1.71410365335599e-06,78.1,12.957,13.133,4689.9,242,,,,
+9500,1,0.00386,1.6503823279524215e-06,66.3,12.957,13.133,4699.5,233,,,,
+9520,1,0.0033,1.5866610025488532e-06,80.5,12.957,13.133,4707.1,224,,,,
+9540,1,0.00423,1.5229396771452845e-06,75.3,12.957,13.133,4715.6,215,,,,
+9560,1,0.00509,1.4592183517417162e-06,53.7,12.957,13.133,4727.6,206,,,,
+9580,1,0.00208,1.3954970263381479e-06,79.1,12.957,13.133,4735.6,197,,,,
+9600,1,0.00108,1.3317757009345794e-06,77.4,12.957,13.133,4743.9,188,,,,
+9620,1,0.00125,1.268054375531011e-06,65.0,12.957,13.133,4753.8,179,,,,
+9640,1,0.00181,1.2043330501274426e-06,79.1,12.957,13.133,4761.9,170,,,,
+9660,1,0.00087,1.1406117247238743e-06,76.9,12.957,13.133,4770.2,161,,,,
+9680,1,0.00296,1.076890399320306e-06,83.4,12.957,13.133,4777.9,152,,,,
+9700,1,0.00018,1.0131690739167374e-06,65.5,12.957,13.133,4787.6,143,,,,
+9720,1,0.00115,9.494477485131691e-07,61.3,12.957,13.133,4798.1,134,,,,
+9740,1,0.01166,8.857264231096007e-07,63.4,12.957,13.133,4808.2,125,,,,
+9760,1,0.00087,8.220050977060323e-07,61.4,12.957,13.133,4818.6,116,,,,
+9780,1,0.00547,7.582837723024639e-07,87.1,12.957,13.133,4825.9,107,,,,
+9800,1,0.00145,6.945624468988954e-07,80.8,12.957,13.133,4833.9,98,,,,
+9820,1,0.00435,6.308411214953271e-07,64.6,12.957,13.133,4843.8,89,,,,
+9840,1,0.00649,5.671197960917587e-07,73.4,12.957,13.133,4852.5,80,,,,
+9860,1,0.00755,5.033984706881903e-07,58.4,12.957,13.133,4863.4,71,,,,
+9880,1,0.00295,4.396771452846219e-07,79.1,12.957,13.133,4871.5,62,,,,
+9900,1,0.00268,3.7595581988105356e-07,58.3,12.957,13.133,4882.5,53,,,,
+9920,1,0.00231,3.1223449447748516e-07,67.4,12.957,13.133,4892.0,44,,,,
+9940,1,0.00417,2.4851316907391675e-07,97.6,12.957,13.133,4898.5,35,,,,
+9960,1,0.00234,1.8479184367034835e-07,62.9,12.957,13.133,4908.7,26,,,,
+9980,1,0.00476,1.2107051826677997e-07,73.8,12.957,13.133,4917.4,17,,,,
+10000,1,0.00142,5.7349192863211556e-08,66.2,12.957,13.133,4927.1,8,0.02981,0.99511,0.99984,0.98616
+10018,1,0.0069,0.0,96.9,12.957,13.133,4985.5,0,0.02979,0.99511,0.99984,0.98616
diff --git a/presentation/checkpoints/finetune/ft-scratch/epochs.csv b/presentation/checkpoints/finetune/ft-scratch/epochs.csv
new file mode 100644
index 0000000000000000000000000000000000000000..99ca9da2f8bf27c08a4ed1100114240c1ee23612
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-scratch/epochs.csv
@@ -0,0 +1,8 @@
+epoch,step,is_epoch_end,n,val_loss,accuracy,precision,recall,f1,roc_auc,pr_auc,tn,fp,fn,tp,threshold,f1_optimal_threshold,f1_at_optimal_threshold,f1_unwanted,n_3way,accuracy_3way,macro_f1_3way,precision_legitimate,recall_legitimate,f1_legitimate,precision_phishing,recall_phishing,f1_phishing,precision_spam,recall_spam,f1_spam,cm3_ll,cm3_lp,cm3_ls,cm3_pl,cm3_pp,cm3_ps,cm3_sl,cm3_sp,cm3_ss,n_unwanted_3way,pvs_phish_as_phish,pvs_phish_as_spam,pvs_spam_as_phish,pvs_spam_as_spam,eval_seconds,is_best
+0,1250,0,15284,0.11794878421139261,0.9651923580214603,0.9571731949032563,0.979594300893504,0.9682539682539683,0.9926827837025348,0.9927681529502044,6639,363,169,8113,0.5,0.7329260110855103,0.9708280485595215,0.9682539682539683,8105,0.9466995681677977,0.863835891081873,0.9958083832335329,0.950014281633819,0.9723724601666422,0.7878787878787878,0.8411764705882353,0.813655761024182,0.692090395480226,0.963302752293578,0.8054794520547945,6652,62,288,15,286,39,13,15,735,1103,296,44,15,748,46.2,1
+0,2500,0,15284,0.08354498752512504,0.974090552211463,0.9627934272300469,0.9904612412460758,0.976431377216998,0.9961333278289897,0.9958895828159741,6685,317,79,8203,0.5,0.8504968285560608,0.9808295153122739,0.976431377216998,8105,0.9576804441702653,0.8860964282715131,0.9991050119331742,0.956583833190517,0.9773821683934043,0.7481662591687042,0.9,0.8170894526034713,0.7641129032258065,0.9934469200524246,0.8638176638176638,6698,102,202,2,306,32,4,1,758,1103,308,32,1,762,46.6,1
+0,3750,0,15284,0.05596537321018193,0.9817456163308035,0.989480122324159,0.9766964501328181,0.9830467278361791,0.9982191585513808,0.9981900545086801,6916,86,193,8089,0.5,0.19911107420921326,0.9848430169613859,0.9830467278361791,8105,0.9835903763109192,0.9531802891108522,0.9936845127027415,0.988717509283062,0.9911947884601617,0.9005524861878453,0.9588235294117647,0.9287749287749287,0.9317010309278351,0.9475753604193972,0.9395711500974658,6923,33,46,7,326,7,37,3,723,1103,333,7,4,759,47.5,1
+0,5000,0,15284,0.0459990399540859,0.9863255692227165,0.9852145690587811,0.9896160347742091,0.9874103969640383,0.9985293210805813,0.9986374826367502,6879,123,86,8196,0.5,0.6643209457397461,0.9881103265133684,0.9874103969640383,8105,0.982850092535472,0.9565657187578056,0.998260113092649,0.9832904884318766,0.9907187567450896,0.9344729344729344,0.9647058823529412,0.9493487698986975,0.8786464410735122,0.9868938401048493,0.9296296296296296,6885,22,95,3,328,9,9,1,753,1103,330,10,1,762,47.5,1
+0,5009,1,15284,0.04709533672780773,0.9863909971211725,0.9863855421686747,0.9885293407389519,0.9874562778916898,0.9984844430897414,0.998618660839301,6889,113,95,8187,0.5,0.6013138890266418,0.9878557186876926,0.9874562778916898,8105,0.9832202344231955,0.9537815709229355,0.9979745370370371,0.9851471008283348,0.991519333045853,0.9384164222873901,0.9411764705882353,0.9397944199706314,0.8814553990610329,0.9842726081258192,0.930030959752322,6898,20,84,3,320,17,11,1,751,1103,322,18,1,762,48.1,1
+1,6250,0,15284,0.05466614213953215,0.9849515833551427,0.9921760391198045,0.9799565322385897,0.9860284291094642,0.9984346418841521,0.9984158381680155,6938,64,166,8116,0.5,0.1293526440858841,0.9877511615277861,0.9860284291094642,8105,0.9875385564466379,0.9648436699653399,0.995551729085952,0.9908597543558983,0.9932002004151457,0.9291784702549575,0.9647058823529412,0.9466089466089466,0.9425287356321839,0.9672346002621232,0.9547218628719275,6938,24,40,7,328,5,24,1,738,1103,335,5,1,762,47.9,0
+1,7500,0,15284,0.05807053193048294,0.9835121695891128,0.9928799410753744,0.9765757063511229,0.9846603360116873,0.9983329960370794,0.998434659179045,6944,58,194,8088,0.5,0.16576719284057617,0.9875885451353151,0.9846603360116873,8105,0.9856878470080197,0.961268841306067,0.9925682435329427,0.991859468723222,0.9922137295521108,0.9295774647887324,0.9705882352941176,0.9496402877697842,0.9482071713147411,0.9357798165137615,0.941952506596306,6945,24,33,4,330,6,48,1,714,1103,333,7,1,762,48.5,0
diff --git a/presentation/checkpoints/finetune/ft-scratch/metrics.csv b/presentation/checkpoints/finetune/ft-scratch/metrics.csv
new file mode 100644
index 0000000000000000000000000000000000000000..59a3c92ed94ba0166943464091da4812d604fa0c
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-scratch/metrics.csv
@@ -0,0 +1,380 @@
+step,epoch,loss,lr,examples_per_s,max_alloc_gb,max_reserved_gb,elapsed_s,eta_s,val_loss,val_f1,val_roc_auc,val_macro_f1_3way
+20,0,0.80568,1.0465116279069768e-06,63.2,12.957,13.131,10.1,5061,,,,
+40,0,0.74627,2.043189368770764e-06,67.3,12.957,13.131,19.6,4899,,,,
+60,0,0.7428,3.0398671096345517e-06,86.0,12.957,13.131,27.1,4495,,,,
+80,0,0.73713,4.036544850498338e-06,96.4,12.957,13.131,33.7,4189,,,,
+100,0,0.68562,5.033222591362126e-06,84.7,12.957,13.131,41.3,4094,,,,
+120,0,0.66308,6.029900332225914e-06,89.8,12.957,13.131,48.4,3993,,,,
+140,0,0.56481,7.026578073089702e-06,76.2,12.957,13.131,56.8,4008,,,,
+160,0,0.51728,8.023255813953488e-06,88.9,12.957,13.131,64.0,3944,,,,
+180,0,0.56358,9.019933554817277e-06,69.2,12.957,13.131,73.3,4004,,,,
+200,0,0.59846,1.0016611295681064e-05,108.7,12.957,13.131,79.1,3885,,,,
+220,0,0.44712,1.1013289036544852e-05,65.9,12.957,13.131,88.9,3957,,,,
+240,0,0.32956,1.2009966777408639e-05,85.8,12.957,13.131,96.3,3924,,,,
+260,0,0.36163,1.3006644518272425e-05,73.1,12.957,13.131,105.1,3943,,,,
+280,0,0.57705,1.4003322259136214e-05,70.5,12.957,13.131,114.2,3970,,,,
+300,0,0.39596,1.5e-05,73.4,12.957,13.131,122.9,3980,,,,
+320,0,0.45832,1.5996677740863787e-05,69.1,12.957,13.131,132.1,4005,,,,
+340,0,0.59169,1.6993355481727575e-05,68.3,12.957,13.131,141.5,4028,,,,
+360,0,0.44385,1.7990033222591364e-05,100.3,12.957,13.131,147.9,3968,,,,
+380,0,0.50835,1.898671096345515e-05,84.3,12.957,13.131,155.5,3944,,,,
+400,0,0.28143,1.9983388704318937e-05,72.9,12.957,13.131,164.3,3950,,,,
+420,0,0.46017,2.0980066445182726e-05,92.2,12.957,13.131,171.2,3913,,,,
+440,0,0.47416,2.197674418604651e-05,65.2,12.957,13.131,181.0,3941,,,,
+460,0,0.47099,2.29734219269103e-05,100.7,12.957,13.131,187.4,3893,,,,
+480,0,0.28938,2.3970099667774087e-05,73.9,12.957,13.131,196.0,3895,,,,
+500,0,0.2715,2.4966777408637872e-05,69.8,12.957,13.131,205.2,3906,,,,
+520,0,0.25174,2.596345514950166e-05,77.1,12.957,13.133,213.5,3900,,,,
+540,0,0.26101,2.696013289036545e-05,84.4,12.957,13.133,221.1,3880,,,,
+560,0,0.28714,2.7956810631229237e-05,75.3,12.957,13.133,229.6,3878,,,,
+580,0,0.29531,2.8953488372093026e-05,71.4,12.957,13.133,238.5,3882,,,,
+600,0,0.33033,2.9950166112956814e-05,79.4,12.957,13.133,246.6,3871,,,,
+620,0,0.33951,2.9942650807136788e-05,95.5,12.957,13.133,253.3,3840,,,,
+640,0,0.20804,2.9878929481733223e-05,96.3,12.957,13.133,260.0,3809,,,,
+660,0,0.25322,2.981520815632965e-05,56.1,12.957,13.133,271.4,3848,,,,
+680,0,0.25769,2.9751486830926084e-05,75.5,12.957,13.133,279.8,3843,,,,
+700,0,0.27089,2.9687765505522515e-05,79.1,12.957,13.133,287.9,3833,,,,
+720,0,0.25447,2.9624044180118947e-05,59.8,12.957,13.133,298.2,3851,,,,
+740,0,0.16905,2.956032285471538e-05,82.4,12.957,13.133,306.0,3836,,,,
+760,0,0.27946,2.949660152931181e-05,84.5,12.957,13.133,313.6,3820,,,,
+780,0,0.21536,2.9432880203908243e-05,86.6,12.957,13.133,320.9,3801,,,,
+800,0,0.16329,2.936915887850467e-05,74.2,12.957,13.133,329.6,3797,,,,
+820,0,0.17881,2.9305437553101107e-05,70.3,12.957,13.133,338.7,3799,,,,
+840,0,0.24714,2.924171622769754e-05,68.9,12.957,13.133,348.0,3802,,,,
+860,0,0.20141,2.9177994902293967e-05,94.9,12.957,13.133,354.7,3777,,,,
+880,0,0.13907,2.91142735768904e-05,74.7,12.957,13.133,363.3,3772,,,,
+900,0,0.22389,2.905055225148683e-05,76.4,12.957,13.133,371.7,3765,,,,
+920,0,0.15485,2.8986830926083263e-05,101.3,12.957,13.133,378.0,3738,,,,
+940,0,0.1859,2.8923109600679695e-05,74.1,12.957,13.133,386.6,3734,,,,
+960,0,0.12477,2.8859388275276127e-05,83.5,12.957,13.133,394.3,3720,,,,
+980,0,0.28579,2.8795666949872555e-05,82.9,12.957,13.133,402.0,3707,,,,
+1000,0,0.1478,2.873194562446899e-05,68.2,12.957,13.133,411.4,3710,,,,
+1020,0,0.16306,2.8668224299065423e-05,73.7,12.957,13.133,420.1,3706,,,,
+1040,0,0.19406,2.860450297366185e-05,69.9,12.957,13.133,429.2,3705,,,,
+1060,0,0.08903,2.8540781648258283e-05,77.0,12.957,13.133,437.5,3698,,,,
+1080,0,0.12314,2.8477060322854715e-05,65.7,12.957,13.133,447.3,3702,,,,
+1100,0,0.15364,2.841333899745115e-05,73.0,12.957,13.133,456.0,3697,,,,
+1120,0,0.08069,2.834961767204758e-05,74.9,12.957,13.133,464.6,3691,,,,
+1140,0,0.1096,2.828589634664401e-05,77.7,12.957,13.133,472.8,3682,,,,
+1160,0,0.13978,2.8222175021240443e-05,86.3,12.957,13.133,480.2,3667,,,,
+1180,0,0.11772,2.8158453695836875e-05,89.7,12.957,13.133,487.4,3650,,,,
+1200,0,0.14603,2.8094732370433307e-05,80.1,12.957,13.133,495.3,3640,,,,
+1220,0,0.09003,2.803101104502974e-05,88.3,12.957,13.133,502.6,3624,,,,
+1240,0,0.11503,2.7967289719626167e-05,63.8,12.957,13.133,512.6,3629,,,,
+1250,0,0.18877,2.7935429056924385e-05,97.7,12.957,13.133,515.9,3619,0.11795,0.96825,0.99268,0.86384
+1260,0,0.11984,2.79035683942226e-05,100.2,12.957,13.133,571.5,3608,,,,
+1280,0,0.11238,2.7839847068819034e-05,65.7,12.957,13.133,581.2,3610,,,,
+1300,0,0.14134,2.7776125743415463e-05,77.1,12.957,13.133,589.5,3602,,,,
+1320,0,0.10874,2.7712404418011895e-05,71.6,12.957,13.133,598.4,3598,,,,
+1340,0,0.08093,2.7648683092608327e-05,82.5,12.957,13.133,606.2,3587,,,,
+1360,0,0.13465,2.758496176720476e-05,69.4,12.957,13.133,615.4,3584,,,,
+1380,0,0.1184,2.752124044180119e-05,71.1,12.957,13.133,624.4,3581,,,,
+1400,0,0.10181,2.7457519116397622e-05,67.4,12.957,13.133,633.9,3580,,,,
+1420,0,0.11927,2.739379779099405e-05,73.2,12.957,13.133,642.7,3574,,,,
+1440,0,0.12675,2.7330076465590486e-05,69.7,12.957,13.133,651.8,3571,,,,
+1460,0,0.16016,2.7266355140186918e-05,64.6,12.957,13.133,661.7,3572,,,,
+1480,0,0.09141,2.720263381478335e-05,69.4,12.957,13.133,671.0,3569,,,,
+1500,0,0.11995,2.713891248937978e-05,78.8,12.957,13.133,679.1,3559,,,,
+1520,0,0.13537,2.707519116397621e-05,75.5,12.957,13.133,687.6,3551,,,,
+1540,0,0.14127,2.7011469838572646e-05,76.8,12.957,13.133,695.9,3543,,,,
+1560,0,0.08485,2.6947748513169074e-05,85.0,12.957,13.133,703.4,3530,,,,
+1580,0,0.11416,2.6884027187765506e-05,66.6,12.957,13.133,713.0,3528,,,,
+1600,0,0.16446,2.6820305862361938e-05,77.8,12.957,13.133,721.3,3519,,,,
+1620,0,0.11242,2.675658453695837e-05,66.9,12.957,13.133,730.8,3517,,,,
+1640,0,0.11405,2.6692863211554802e-05,74.7,12.957,13.133,739.4,3510,,,,
+1660,0,0.08504,2.6629141886151234e-05,80.7,12.957,13.133,747.3,3499,,,,
+1680,0,0.11291,2.6565420560747662e-05,65.9,12.957,13.133,757.0,3497,,,,
+1700,0,0.11074,2.6501699235344094e-05,61.6,12.957,13.133,767.4,3499,,,,
+1720,0,0.11307,2.643797790994053e-05,75.8,12.957,13.133,775.8,3490,,,,
+1740,0,0.0788,2.6374256584536958e-05,73.9,12.957,13.133,784.5,3483,,,,
+1760,0,0.09768,2.631053525913339e-05,70.7,12.957,13.133,793.6,3478,,,,
+1780,0,0.13344,2.6246813933729822e-05,80.9,12.957,13.133,801.5,3467,,,,
+1800,0,0.10509,2.6183092608326254e-05,75.7,12.957,13.133,809.9,3459,,,,
+1820,0,0.06411,2.6119371282922686e-05,77.6,12.957,13.133,818.2,3449,,,,
+1840,0,0.10486,2.6055649957519118e-05,101.4,12.957,13.133,824.5,3432,,,,
+1860,0,0.09431,2.599192863211555e-05,83.2,12.957,13.133,832.2,3420,,,,
+1880,0,0.09652,2.5928207306711978e-05,61.7,12.957,13.133,842.5,3420,,,,
+1900,0,0.07351,2.5864485981308413e-05,81.2,12.957,13.133,850.4,3410,,,,
+1920,0,0.09196,2.5800764655904845e-05,72.1,12.957,13.133,859.3,3403,,,,
+1940,0,0.10419,2.5737043330501274e-05,86.1,12.957,13.133,866.7,3391,,,,
+1960,0,0.05586,2.5673322005097706e-05,95.4,12.957,13.133,873.4,3376,,,,
+1980,0,0.12768,2.560960067969414e-05,71.2,12.957,13.133,882.4,3370,,,,
+2000,0,0.05465,2.554587935429057e-05,75.1,12.957,13.133,891.0,3362,,,,
+2020,0,0.09087,2.5482158028887e-05,69.9,12.957,13.133,900.1,3357,,,,
+2040,0,0.09986,2.5418436703483433e-05,76.1,12.957,13.133,908.5,3348,,,,
+2060,0,0.07248,2.5354715378079862e-05,66.5,12.957,13.133,918.1,3345,,,,
+2080,0,0.10174,2.5290994052676297e-05,66.4,12.957,13.133,927.8,3341,,,,
+2100,0,0.06688,2.522727272727273e-05,71.6,12.957,13.133,936.7,3334,,,,
+2120,0,0.08858,2.5163551401869158e-05,66.6,12.957,13.133,946.3,3330,,,,
+2140,0,0.09811,2.509983007646559e-05,57.4,12.957,13.133,957.5,3332,,,,
+2160,0,0.06774,2.5036108751062025e-05,69.7,12.957,13.133,966.7,3326,,,,
+2180,0,0.09665,2.4972387425658453e-05,75.6,12.957,13.133,975.1,3318,,,,
+2200,0,0.0665,2.4908666100254885e-05,76.0,12.957,13.133,983.5,3309,,,,
+2220,0,0.08279,2.4844944774851317e-05,61.5,12.957,13.133,994.0,3307,,,,
+2240,0,0.07478,2.478122344944775e-05,81.7,12.957,13.133,1001.8,3297,,,,
+2260,0,0.08226,2.471750212404418e-05,64.5,12.957,13.133,1011.7,3293,,,,
+2280,0,0.11436,2.4653780798640613e-05,62.9,12.957,13.133,1021.9,3290,,,,
+2300,0,0.08772,2.4590059473237045e-05,71.0,12.957,13.133,1030.9,3284,,,,
+2320,0,0.07195,2.4526338147833473e-05,73.5,12.957,13.133,1039.6,3276,,,,
+2340,0,0.12296,2.446261682242991e-05,83.8,12.957,13.133,1047.3,3264,,,,
+2360,0,0.08304,2.439889549702634e-05,63.1,12.957,13.133,1057.4,3261,,,,
+2380,0,0.07734,2.433517417162277e-05,75.5,12.957,13.133,1065.9,3253,,,,
+2400,0,0.06877,2.42714528462192e-05,78.3,12.957,13.133,1074.0,3243,,,,
+2420,0,0.10848,2.4207731520815636e-05,85.9,12.957,13.133,1081.5,3231,,,,
+2440,0,0.03477,2.4144010195412065e-05,83.0,12.957,13.133,1089.2,3220,,,,
+2460,0,0.17972,2.4080288870008497e-05,62.1,12.957,13.133,1099.5,3217,,,,
+2480,0,0.10408,2.401656754460493e-05,62.8,12.957,13.133,1109.7,3214,,,,
+2500,0,0.09738,2.3952846219201357e-05,66.9,12.957,13.133,1119.3,3208,0.08354,0.97643,0.99613,0.8861
+2520,0,0.09403,2.3889124893797792e-05,70.4,12.957,13.133,1179.8,3202,,,,
+2540,0,0.09673,2.3825403568394224e-05,66.6,12.957,13.133,1189.4,3196,,,,
+2560,0,0.13906,2.3761682242990653e-05,68.8,12.957,13.133,1198.7,3190,,,,
+2580,0,0.06143,2.3697960917587085e-05,87.3,12.957,13.133,1206.0,3178,,,,
+2600,0,0.08035,2.363423959218352e-05,71.2,12.957,13.133,1215.0,3170,,,,
+2620,0,0.09823,2.3570518266779952e-05,68.2,12.957,13.133,1224.4,3164,,,,
+2640,0,0.0843,2.350679694137638e-05,82.3,12.957,13.133,1232.2,3154,,,,
+2660,0,0.0496,2.3443075615972812e-05,93.3,12.957,13.133,1239.0,3140,,,,
+2680,0,0.072,2.3379354290569244e-05,77.3,12.957,13.133,1247.3,3131,,,,
+2700,0,0.06786,2.3315632965165676e-05,68.0,12.957,13.133,1256.7,3125,,,,
+2720,0,0.09985,2.3251911639762108e-05,82.3,12.957,13.133,1264.5,3114,,,,
+2740,0,0.10277,2.318819031435854e-05,69.1,12.957,13.133,1273.7,3108,,,,
+2760,0,0.05185,2.312446898895497e-05,72.9,12.957,13.133,1282.5,3100,,,,
+2780,0,0.10491,2.3060747663551404e-05,77.3,12.957,13.133,1290.8,3091,,,,
+2800,0,0.08649,2.2997026338147836e-05,63.8,12.957,13.133,1300.8,3086,,,,
+2820,0,0.05712,2.2933305012744264e-05,89.6,12.957,13.133,1308.0,3074,,,,
+2840,0,0.08216,2.2869583687340696e-05,62.5,12.957,13.133,1318.2,3069,,,,
+2860,0,0.0779,2.2805862361937128e-05,60.4,12.957,13.133,1328.8,3066,,,,
+2880,0,0.03263,2.274214103653356e-05,74.2,12.957,13.133,1337.4,3058,,,,
+2900,0,0.04114,2.2678419711129992e-05,76.5,12.957,13.133,1345.8,3049,,,,
+2920,0,0.10578,2.2614698385726424e-05,73.0,12.957,13.133,1354.6,3040,,,,
+2940,0,0.08776,2.2550977060322852e-05,54.9,12.957,13.133,1366.2,3039,,,,
+2960,0,0.09051,2.2487255734919288e-05,71.5,12.957,13.133,1375.2,3032,,,,
+2980,0,0.04238,2.242353440951572e-05,89.0,12.957,13.133,1382.4,3020,,,,
+3000,0,0.05557,2.235981308411215e-05,67.4,12.957,13.133,1391.8,3013,,,,
+3020,0,0.05912,2.229609175870858e-05,87.5,12.957,13.133,1399.2,3002,,,,
+3040,0,0.05564,2.2232370433305012e-05,76.5,12.957,13.133,1407.5,2993,,,,
+3060,0,0.07358,2.2168649107901447e-05,80.5,12.957,13.133,1415.5,2983,,,,
+3080,0,0.07145,2.2104927782497876e-05,67.4,12.957,13.133,1425.0,2976,,,,
+3100,0,0.06806,2.2041206457094308e-05,65.8,12.957,13.133,1434.7,2970,,,,
+3120,0,0.04876,2.197748513169074e-05,70.5,12.957,13.133,1443.8,2963,,,,
+3140,0,0.06113,2.191376380628717e-05,69.9,12.957,13.133,1452.9,2955,,,,
+3160,0,0.04846,2.1850042480883604e-05,81.9,12.957,13.133,1460.7,2945,,,,
+3180,0,0.05859,2.1786321155480035e-05,70.7,12.957,13.133,1469.8,2937,,,,
+3200,0,0.09876,2.1722599830076464e-05,69.1,12.957,13.133,1479.1,2930,,,,
+3220,0,0.0691,2.16588785046729e-05,72.2,12.957,13.133,1487.9,2922,,,,
+3240,0,0.02506,2.159515717926933e-05,65.9,12.957,13.133,1497.6,2916,,,,
+3260,0,0.04611,2.153143585386576e-05,67.9,12.957,13.133,1507.0,2909,,,,
+3280,0,0.07513,2.146771452846219e-05,64.4,12.957,13.133,1517.0,2903,,,,
+3300,0,0.09106,2.1403993203058624e-05,86.1,12.957,13.133,1524.4,2892,,,,
+3320,0,0.07776,2.1340271877655055e-05,73.9,12.957,13.133,1533.1,2884,,,,
+3340,0,0.0416,2.1276550552251487e-05,101.7,12.957,13.133,1539.4,2870,,,,
+3360,0,0.0631,2.121282922684792e-05,66.7,12.957,13.133,1549.0,2864,,,,
+3380,0,0.07246,2.1149107901444348e-05,67.4,12.957,13.133,1558.5,2857,,,,
+3400,0,0.0831,2.1085386576040783e-05,79.4,12.957,13.133,1566.5,2847,,,,
+3420,0,0.08234,2.1021665250637215e-05,72.6,12.957,13.133,1575.3,2839,,,,
+3440,0,0.08466,2.0957943925233647e-05,68.2,12.957,13.133,1584.7,2832,,,,
+3460,0,0.05655,2.0894222599830075e-05,72.9,12.957,13.133,1593.5,2824,,,,
+3480,0,0.06352,2.0830501274426507e-05,57.2,12.957,13.133,1604.7,2820,,,,
+3500,0,0.04625,2.0766779949022943e-05,84.7,12.957,13.133,1612.2,2809,,,,
+3520,0,0.03823,2.070305862361937e-05,94.6,12.957,13.133,1619.0,2797,,,,
+3540,0,0.08285,2.0639337298215803e-05,77.3,12.957,13.133,1627.3,2788,,,,
+3560,0,0.06093,2.0575615972812235e-05,80.4,12.957,13.133,1635.2,2778,,,,
+3580,0,0.05367,2.0511894647408667e-05,75.7,12.957,13.133,1643.7,2769,,,,
+3600,0,0.06575,2.04481733220051e-05,67.5,12.957,13.133,1653.2,2762,,,,
+3620,0,0.11097,2.038445199660153e-05,69.8,12.957,13.133,1662.4,2755,,,,
+3640,0,0.05022,2.032073067119796e-05,96.3,12.957,13.133,1669.0,2743,,,,
+3660,0,0.06598,2.025700934579439e-05,76.4,12.957,13.133,1677.4,2734,,,,
+3680,0,0.07757,2.0193288020390827e-05,69.8,12.957,13.133,1686.6,2726,,,,
+3700,0,0.06255,2.0129566694987255e-05,86.4,12.957,13.133,1694.0,2715,,,,
+3720,0,0.04166,2.0065845369583687e-05,69.6,12.957,13.133,1703.1,2708,,,,
+3740,0,0.09515,2.000212404418012e-05,68.2,12.957,13.133,1712.5,2700,,,,
+3750,0,0.11501,1.9970263381478337e-05,76.0,12.957,13.133,1716.7,2696,0.05597,0.98305,0.99822,0.95318
+3760,0,0.04873,1.993840271877655e-05,72.9,12.957,13.133,1773.4,2692,,,,
+3780,0,0.08283,1.9874681393372983e-05,77.5,12.957,13.133,1781.6,2683,,,,
+3800,0,0.0748,1.9810960067969415e-05,72.8,12.957,13.133,1790.4,2674,,,,
+3820,0,0.11127,1.9747238742565847e-05,69.6,12.957,13.133,1799.6,2667,,,,
+3840,0,0.06913,1.9683517417162275e-05,77.7,12.957,13.133,1807.9,2658,,,,
+3860,0,0.03809,1.961979609175871e-05,68.0,12.957,13.133,1817.3,2650,,,,
+3880,0,0.04129,1.9556074766355142e-05,75.9,12.957,13.133,1825.7,2641,,,,
+3900,0,0.08388,1.949235344095157e-05,70.7,12.957,13.133,1834.8,2633,,,,
+3920,0,0.09063,1.9428632115548003e-05,70.6,12.957,13.133,1843.8,2626,,,,
+3940,0,0.08573,1.9364910790144438e-05,62.0,12.957,13.133,1854.1,2620,,,,
+3960,0,0.0597,1.9301189464740867e-05,77.4,12.957,13.133,1862.4,2610,,,,
+3980,0,0.13358,1.92374681393373e-05,84.8,12.957,13.133,1870.0,2600,,,,
+4000,0,0.07208,1.917374681393373e-05,69.3,12.957,13.133,1879.2,2593,,,,
+4020,0,0.08458,1.9110025488530162e-05,71.7,12.957,13.133,1888.1,2584,,,,
+4040,0,0.08685,1.9046304163126594e-05,64.7,12.957,13.133,1898.0,2578,,,,
+4060,0,0.0777,1.8982582837723026e-05,76.0,12.957,13.133,1906.5,2569,,,,
+4080,0,0.06219,1.8918861512319455e-05,59.3,12.957,13.133,1917.2,2563,,,,
+4100,0,0.03318,1.8855140186915887e-05,73.5,12.957,13.133,1925.9,2555,,,,
+4120,0,0.05001,1.8791418861512322e-05,74.8,12.957,13.133,1934.5,2546,,,,
+4140,0,0.09007,1.872769753610875e-05,65.9,12.957,13.133,1944.2,2539,,,,
+4160,0,0.06556,1.8663976210705182e-05,70.5,12.957,13.133,1953.3,2531,,,,
+4180,0,0.07277,1.8600254885301614e-05,65.1,12.957,13.133,1963.1,2524,,,,
+4200,0,0.06252,1.853653355989805e-05,74.1,12.957,13.133,1971.8,2515,,,,
+4220,0,0.05228,1.8472812234494478e-05,95.9,12.957,13.133,1978.4,2504,,,,
+4240,0,0.04207,1.840909090909091e-05,89.9,12.957,13.133,1985.6,2493,,,,
+4260,0,0.11709,1.8345369583687342e-05,59.2,12.957,13.133,1996.4,2488,,,,
+4280,0,0.05393,1.828164825828377e-05,76.9,12.957,13.133,2004.7,2478,,,,
+4300,0,0.09023,1.8217926932880206e-05,60.2,12.957,13.133,2015.3,2472,,,,
+4320,0,0.0745,1.8154205607476638e-05,81.0,12.957,13.133,2023.2,2463,,,,
+4340,0,0.04241,1.8090484282073066e-05,63.2,12.957,13.133,2033.3,2456,,,,
+4360,0,0.07594,1.8026762956669498e-05,61.8,12.957,13.133,2043.7,2450,,,,
+4380,0,0.02425,1.7963041631265933e-05,64.9,12.957,13.133,2053.6,2443,,,,
+4400,0,0.08944,1.7899320305862362e-05,62.3,12.957,13.133,2063.8,2436,,,,
+4420,0,0.07593,1.7835598980458794e-05,69.0,12.957,13.133,2073.1,2428,,,,
+4440,0,0.04351,1.7771877655055226e-05,81.8,12.957,13.133,2080.9,2418,,,,
+4460,0,0.03204,1.7708156329651654e-05,70.9,12.957,13.133,2089.9,2410,,,,
+4480,0,0.05587,1.764443500424809e-05,65.1,12.957,13.133,2099.8,2403,,,,
+4500,0,0.03737,1.758071367884452e-05,73.9,12.957,13.133,2108.4,2394,,,,
+4520,0,0.04455,1.751699235344095e-05,68.2,12.957,13.133,2117.8,2386,,,,
+4540,0,0.0388,1.7453271028037382e-05,78.2,12.957,13.133,2126.0,2377,,,,
+4560,0,0.07318,1.7389549702633817e-05,60.5,12.957,13.133,2136.6,2371,,,,
+4580,0,0.07732,1.732582837723025e-05,63.0,12.957,13.133,2146.7,2364,,,,
+4600,0,0.05311,1.7262107051826678e-05,62.3,12.957,13.133,2157.0,2357,,,,
+4620,0,0.0675,1.719838572642311e-05,67.5,12.957,13.133,2166.5,2349,,,,
+4640,0,0.06562,1.713466440101954e-05,98.3,12.957,13.133,2173.0,2338,,,,
+4660,0,0.07196,1.7070943075615973e-05,64.5,12.957,13.133,2182.9,2331,,,,
+4680,0,0.05662,1.7007221750212405e-05,76.0,12.957,13.133,2191.3,2321,,,,
+4700,0,0.05744,1.6943500424808837e-05,80.1,12.957,13.133,2199.3,2312,,,,
+4720,0,0.08038,1.6879779099405266e-05,63.2,12.957,13.133,2209.4,2305,,,,
+4740,0,0.05095,1.68160577740017e-05,70.9,12.957,13.133,2218.5,2297,,,,
+4760,0,0.098,1.6752336448598133e-05,70.7,12.957,13.133,2227.5,2288,,,,
+4780,0,0.07721,1.668861512319456e-05,84.6,12.957,13.133,2235.1,2278,,,,
+4800,0,0.05332,1.6624893797790993e-05,72.6,12.957,13.133,2243.9,2270,,,,
+4820,0,0.05705,1.6561172472387425e-05,71.4,12.957,13.133,2252.9,2261,,,,
+4840,0,0.05274,1.6497451146983857e-05,58.2,12.957,13.133,2263.9,2255,,,,
+4860,0,0.03311,1.643372982158029e-05,83.0,12.957,13.133,2271.6,2245,,,,
+4880,0,0.03198,1.637000849617672e-05,74.0,12.957,13.133,2280.2,2237,,,,
+4900,0,0.05952,1.630628717077315e-05,62.3,12.957,13.133,2290.5,2229,,,,
+4920,0,0.0442,1.6242565845369585e-05,66.7,12.957,13.133,2300.1,2222,,,,
+4940,0,0.06009,1.6178844519966017e-05,72.0,12.957,13.133,2309.0,2213,,,,
+4960,0,0.03121,1.611512319456245e-05,62.4,12.957,13.133,2319.2,2206,,,,
+4980,0,0.05693,1.6051401869158877e-05,65.3,12.957,13.133,2329.0,2198,,,,
+5000,0,0.06858,1.5987680543755312e-05,71.9,12.957,13.133,2337.9,2190,0.046,0.98741,0.99853,0.95657
+5009,0,0.07588,1.5959005947323706e-05,97.3,12.957,13.133,2393.6,2185,0.0471,0.98746,0.99848,0.95378
+5020,1,0.02634,1.5923959218351744e-05,82.3,12.957,13.133,2451.3,2180,,,,
+5040,1,0.0553,1.5860237892948173e-05,75.3,12.957,13.133,2459.8,2171,,,,
+5060,1,0.03058,1.5796516567544605e-05,72.8,12.957,13.133,2468.6,2162,,,,
+5080,1,0.04746,1.5732795242141037e-05,62.8,12.957,13.133,2478.8,2155,,,,
+5100,1,0.05239,1.566907391673747e-05,85.5,12.957,13.133,2486.3,2145,,,,
+5120,1,0.04319,1.56053525913339e-05,75.0,12.957,13.133,2494.8,2136,,,,
+5140,1,0.02477,1.5541631265930333e-05,66.5,12.957,13.133,2504.5,2128,,,,
+5160,1,0.03801,1.547790994052676e-05,81.3,12.957,13.133,2512.3,2118,,,,
+5180,1,0.0518,1.5414188615123196e-05,108.2,12.957,13.133,2518.3,2107,,,,
+5200,1,0.03274,1.5350467289719628e-05,67.4,12.957,13.133,2527.7,2099,,,,
+5220,1,0.02178,1.5286745964316057e-05,80.4,12.957,13.133,2535.7,2090,,,,
+5240,1,0.0481,1.5223024638912489e-05,61.6,12.957,13.133,2546.1,2083,,,,
+5260,1,0.09417,1.5159303313508922e-05,74.2,12.957,13.133,2554.7,2074,,,,
+5280,1,0.0492,1.509558198810535e-05,71.6,12.957,13.133,2563.7,2065,,,,
+5300,1,0.03453,1.5031860662701784e-05,60.9,12.957,13.133,2574.2,2058,,,,
+5320,1,0.01756,1.4968139337298216e-05,57.2,12.957,13.133,2585.4,2052,,,,
+5340,1,0.04211,1.4904418011894648e-05,70.9,12.957,13.133,2594.4,2043,,,,
+5360,1,0.06567,1.4840696686491078e-05,71.0,12.957,13.133,2603.4,2035,,,,
+5380,1,0.02769,1.4776975361087512e-05,62.0,12.957,13.133,2613.7,2027,,,,
+5400,1,0.03959,1.4713254035683942e-05,62.0,12.957,13.133,2624.1,2020,,,,
+5420,1,0.02799,1.4649532710280374e-05,68.0,12.957,13.133,2633.5,2012,,,,
+5440,1,0.08605,1.4585811384876806e-05,68.6,12.957,13.133,2642.8,2003,,,,
+5460,1,0.05379,1.4522090059473236e-05,68.6,12.957,13.133,2652.1,1995,,,,
+5480,1,0.03286,1.445836873406967e-05,67.7,12.957,13.133,2661.6,1987,,,,
+5500,1,0.0666,1.43946474086661e-05,74.8,12.957,13.133,2670.1,1978,,,,
+5520,1,0.03685,1.4330926083262532e-05,87.6,12.957,13.133,2677.4,1968,,,,
+5540,1,0.02861,1.4267204757858964e-05,62.7,12.957,13.133,2687.6,1961,,,,
+5560,1,0.05208,1.4203483432455396e-05,81.1,12.957,13.133,2695.5,1951,,,,
+5580,1,0.04012,1.4139762107051826e-05,125.5,12.957,13.133,2700.6,1939,,,,
+5600,1,0.0342,1.407604078164826e-05,82.3,12.957,13.133,2708.4,1930,,,,
+5620,1,0.0458,1.401231945624469e-05,70.0,12.957,13.133,2717.6,1922,,,,
+5640,1,0.02065,1.3948598130841122e-05,72.1,12.957,13.133,2726.4,1913,,,,
+5660,1,0.05034,1.3884876805437554e-05,62.6,12.957,13.133,2736.7,1905,,,,
+5680,1,0.02209,1.3821155480033984e-05,72.9,12.957,13.133,2745.4,1897,,,,
+5700,1,0.05621,1.3757434154630418e-05,68.3,12.957,13.133,2754.8,1888,,,,
+5720,1,0.03269,1.3693712829226848e-05,79.3,12.957,13.133,2762.9,1879,,,,
+5740,1,0.02551,1.362999150382328e-05,90.1,12.957,13.133,2770.0,1869,,,,
+5760,1,0.00555,1.3566270178419712e-05,65.9,12.957,13.133,2779.7,1861,,,,
+5780,1,0.064,1.3502548853016144e-05,66.2,12.957,13.133,2789.4,1853,,,,
+5800,1,0.03932,1.3438827527612575e-05,79.7,12.957,13.133,2797.4,1844,,,,
+5820,1,0.02253,1.3375106202209006e-05,61.3,12.957,13.133,2807.8,1836,,,,
+5840,1,0.04029,1.3311384876805438e-05,71.8,12.957,13.133,2816.7,1828,,,,
+5860,1,0.03217,1.324766355140187e-05,72.0,12.957,13.133,2825.6,1819,,,,
+5880,1,0.05183,1.3183942225998301e-05,63.9,12.957,13.133,2835.6,1811,,,,
+5900,1,0.05542,1.3120220900594732e-05,71.5,12.957,13.133,2844.6,1802,,,,
+5920,1,0.03102,1.3056499575191165e-05,66.3,12.957,13.133,2854.3,1794,,,,
+5940,1,0.0714,1.2992778249787595e-05,64.5,12.957,13.133,2864.2,1786,,,,
+5960,1,0.04789,1.2929056924384027e-05,62.8,12.957,13.133,2874.4,1779,,,,
+5980,1,0.06999,1.286533559898046e-05,65.6,12.957,13.133,2884.1,1770,,,,
+6000,1,0.04846,1.2801614273576891e-05,61.1,12.957,13.133,2894.6,1763,,,,
+6020,1,0.03049,1.2737892948173323e-05,75.5,12.957,13.133,2903.1,1754,,,,
+6040,1,0.00432,1.2674171622769753e-05,74.4,12.957,13.133,2911.7,1745,,,,
+6060,1,0.05036,1.2610450297366185e-05,61.9,12.957,13.133,2922.0,1737,,,,
+6080,1,0.03953,1.2546728971962617e-05,78.8,12.957,13.133,2930.1,1728,,,,
+6100,1,0.02367,1.2483007646559049e-05,63.8,12.957,13.133,2940.2,1720,,,,
+6120,1,0.01284,1.241928632115548e-05,73.2,12.957,13.133,2948.9,1711,,,,
+6140,1,0.05799,1.2355564995751913e-05,95.2,12.957,13.133,2955.6,1701,,,,
+6160,1,0.01128,1.2291843670348343e-05,67.2,12.957,13.133,2965.1,1693,,,,
+6180,1,0.04712,1.2228122344944777e-05,71.4,12.957,13.133,2974.1,1684,,,,
+6200,1,0.03532,1.2164401019541207e-05,71.0,12.957,13.133,2983.1,1676,,,,
+6220,1,0.04701,1.2100679694137637e-05,80.2,12.957,13.133,2991.1,1666,,,,
+6240,1,0.04175,1.203695836873407e-05,80.8,12.957,13.133,2999.0,1657,,,,
+6250,1,0.04467,1.2005097706032285e-05,48.7,12.957,13.133,3005.6,1654,0.05467,0.98603,0.99843,0.96484
+6260,1,0.05329,1.1973237043330501e-05,71.8,12.957,13.133,3061.6,1650,,,,
+6280,1,0.02926,1.1909515717926933e-05,104.5,12.957,13.133,3067.7,1639,,,,
+6300,1,0.09557,1.1845794392523365e-05,67.6,12.957,13.133,3077.2,1631,,,,
+6320,1,0.03173,1.1782073067119797e-05,63.4,12.957,13.133,3087.3,1623,,,,
+6340,1,0.02212,1.1718351741716227e-05,79.3,12.957,13.133,3095.4,1614,,,,
+6360,1,0.04183,1.165463041631266e-05,114.1,12.957,13.133,3101.0,1603,,,,
+6380,1,0.03398,1.159090909090909e-05,70.7,12.957,13.133,3110.0,1595,,,,
+6400,1,0.02526,1.1527187765505523e-05,89.2,12.957,13.133,3117.2,1585,,,,
+6420,1,0.05276,1.1463466440101955e-05,67.4,12.957,13.133,3126.7,1577,,,,
+6440,1,0.05771,1.1399745114698385e-05,64.1,12.957,13.133,3136.7,1568,,,,
+6460,1,0.03371,1.1336023789294818e-05,74.3,12.957,13.133,3145.3,1560,,,,
+6480,1,0.01164,1.1272302463891249e-05,72.0,12.957,13.133,3154.2,1551,,,,
+6500,1,0.06465,1.120858113848768e-05,88.2,12.957,13.133,3161.5,1541,,,,
+6520,1,0.03712,1.1144859813084113e-05,77.5,12.957,13.133,3169.7,1532,,,,
+6540,1,0.05561,1.1081138487680544e-05,74.0,12.957,13.133,3178.4,1523,,,,
+6560,1,0.01981,1.1017417162276975e-05,83.7,12.957,13.133,3186.0,1514,,,,
+6580,1,0.05338,1.0953695836873408e-05,69.3,12.957,13.133,3195.3,1506,,,,
+6600,1,0.04691,1.0889974511469838e-05,72.2,12.957,13.133,3204.1,1497,,,,
+6620,1,0.03252,1.082625318606627e-05,59.7,12.957,13.133,3214.8,1489,,,,
+6640,1,0.03197,1.0762531860662702e-05,57.8,12.957,13.133,3225.9,1482,,,,
+6660,1,0.05185,1.0698810535259133e-05,93.7,12.957,13.133,3232.7,1472,,,,
+6680,1,0.06263,1.0635089209855566e-05,75.7,12.957,13.133,3241.2,1463,,,,
+6700,1,0.02125,1.0571367884451996e-05,69.4,12.957,13.133,3250.4,1454,,,,
+6720,1,0.02088,1.0507646559048428e-05,64.3,12.957,13.133,3260.4,1446,,,,
+6740,1,0.03623,1.044392523364486e-05,68.4,12.957,13.133,3269.7,1438,,,,
+6760,1,0.03119,1.0380203908241292e-05,63.7,12.957,13.133,3279.8,1430,,,,
+6780,1,0.02303,1.0316482582837724e-05,65.1,12.957,13.133,3289.6,1421,,,,
+6800,1,0.03927,1.0252761257434154e-05,78.7,12.957,13.133,3297.7,1412,,,,
+6820,1,0.0266,1.0189039932030586e-05,60.4,12.957,13.133,3308.3,1404,,,,
+6840,1,0.0309,1.0125318606627018e-05,86.3,12.957,13.133,3315.7,1395,,,,
+6860,1,0.01276,1.006159728122345e-05,71.1,12.957,13.133,3324.7,1386,,,,
+6880,1,0.04029,9.99787595581988e-06,83.8,12.957,13.133,3332.4,1377,,,,
+6900,1,0.01982,9.934154630416314e-06,83.8,12.957,13.133,3340.0,1368,,,,
+6920,1,0.03481,9.870433305012744e-06,65.2,12.957,13.133,3349.8,1359,,,,
+6940,1,0.04858,9.806711979609176e-06,60.4,12.957,13.133,3360.4,1351,,,,
+6960,1,0.03442,9.742990654205608e-06,85.5,12.957,13.133,3367.9,1342,,,,
+6980,1,0.02365,9.67926932880204e-06,84.6,12.957,13.133,3375.5,1333,,,,
+7000,1,0.03698,9.615548003398472e-06,65.9,12.957,13.133,3385.2,1324,,,,
+7020,1,0.02483,9.551826677994902e-06,83.8,12.957,13.133,3392.8,1315,,,,
+7040,1,0.03178,9.488105352591334e-06,77.8,12.957,13.133,3401.1,1306,,,,
+7060,1,0.03735,9.424384027187766e-06,64.1,12.957,13.133,3411.0,1298,,,,
+7080,1,0.07443,9.360662701784198e-06,74.2,12.957,13.133,3419.7,1289,,,,
+7100,1,0.02627,9.296941376380628e-06,69.6,12.957,13.133,3428.9,1280,,,,
+7120,1,0.03457,9.233220050977061e-06,73.6,12.957,13.133,3437.5,1271,,,,
+7140,1,0.04569,9.169498725573492e-06,67.6,12.957,13.133,3447.0,1263,,,,
+7160,1,0.02134,9.105777400169925e-06,57.2,12.957,13.133,3458.2,1255,,,,
+7180,1,0.02019,9.042056074766356e-06,82.7,12.957,13.133,3465.9,1246,,,,
+7200,1,0.03219,8.978334749362786e-06,57.9,12.957,13.133,3477.0,1238,,,,
+7220,1,0.02055,8.91461342395922e-06,77.9,12.957,13.133,3485.2,1229,,,,
+7240,1,0.00588,8.85089209855565e-06,102.6,12.957,13.133,3491.5,1219,,,,
+7260,1,0.03122,8.787170773152081e-06,57.4,12.957,13.133,3502.6,1211,,,,
+7280,1,0.04576,8.723449447748513e-06,62.9,12.957,13.133,3512.8,1203,,,,
+7300,1,0.02569,8.659728122344945e-06,58.7,12.957,13.133,3523.7,1195,,,,
+7320,1,0.06377,8.596006796941376e-06,86.7,12.957,13.133,3531.1,1186,,,,
+7340,1,0.00984,8.532285471537809e-06,82.7,12.957,13.133,3538.8,1177,,,,
+7360,1,0.02559,8.46856414613424e-06,74.4,12.957,13.133,3547.4,1168,,,,
+7380,1,0.05262,8.404842820730673e-06,85.1,12.957,13.133,3554.9,1159,,,,
+7400,1,0.01546,8.341121495327103e-06,69.6,12.957,13.133,3564.1,1150,,,,
+7420,1,0.02079,8.277400169923533e-06,70.7,12.957,13.133,3573.2,1141,,,,
+7440,1,0.04407,8.213678844519967e-06,65.9,12.957,13.133,3582.9,1133,,,,
+7460,1,0.09019,8.149957519116397e-06,68.0,12.957,13.133,3592.3,1124,,,,
+7480,1,0.06463,8.086236193712829e-06,89.6,12.957,13.133,3599.4,1115,,,,
+7500,1,0.02854,8.022514868309261e-06,75.3,12.957,13.133,3607.9,1106,0.05807,0.98466,0.99833,0.96127
diff --git a/presentation/checkpoints/finetune/ft-synthetic/epochs.csv b/presentation/checkpoints/finetune/ft-synthetic/epochs.csv
new file mode 100644
index 0000000000000000000000000000000000000000..7523ed0e1e813ab220d56dabd49a1611a832f738
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-synthetic/epochs.csv
@@ -0,0 +1,7 @@
+epoch,step,is_epoch_end,n,val_loss,accuracy,precision,recall,f1,roc_auc,pr_auc,tn,fp,fn,tp,threshold,f1_optimal_threshold,f1_at_optimal_threshold,f1_unwanted,n_3way,accuracy_3way,macro_f1_3way,precision_legitimate,recall_legitimate,f1_legitimate,precision_phishing,recall_phishing,f1_phishing,precision_spam,recall_spam,f1_spam,cm3_ll,cm3_lp,cm3_ls,cm3_pl,cm3_pp,cm3_ps,cm3_sl,cm3_sp,cm3_ss,n_unwanted_3way,pvs_phish_as_phish,pvs_phish_as_spam,pvs_spam_as_phish,pvs_spam_as_spam,eval_seconds,is_best
+0,1250,0,15284,0.13253656278508896,0.956163308034546,0.9945426195426196,0.9241729050953876,0.9580673425960696,0.9977272940473557,0.9979187104680562,6960,42,628,7654,0.5,0.030636101961135864,0.9823415578132559,0.9580673425960696,8105,0.9756940160394818,0.9257717181041681,0.9794568735049951,0.9941445301342473,0.9867460486214473,0.9649122807017544,0.8088235294117647,0.88,0.9424964936886395,0.8807339449541285,0.9105691056910569,6961,8,33,57,275,8,89,2,672,1103,321,19,3,760,49.0,1
+0,2500,0,15284,0.048395975874344487,0.9855404344412457,0.9883678662304617,0.9849070272880946,0.9866344118536438,0.9986052730923605,0.9988524994193232,6906,96,125,8157,0.5,0.2712920606136322,0.9869300728783955,0.9866344118536438,8105,0.98470080197409,0.9555283650831455,0.9962519821248378,0.987003713224793,0.9916062845254322,0.8701298701298701,0.9852941176470589,0.9241379310344827,0.9386973180076629,0.963302752293578,0.9508408796895214,6911,47,44,1,335,4,25,3,735,1103,336,4,4,759,48.0,1
+0,3750,0,15284,0.04945425575346074,0.9881575503794818,0.9947477708562354,0.9833373581260565,0.9890096545023984,0.9993127675047271,0.9994189406738688,6959,43,138,8144,0.5,0.10066894441843033,0.9903921687111004,0.9890096545023984,8105,0.9895126465144972,0.970149730859442,0.9955650929899857,0.9938588974578692,0.9947112635791882,0.9875776397515528,0.9352941176470588,0.9607250755287009,0.9369482976040353,0.9737876802096985,0.9550128534704371,6959,3,40,12,318,10,19,1,743,1103,329,11,1,762,48.1,1
+0,5000,0,15284,0.027758434301769195,0.9920177963883801,0.9903846153846154,0.9949287611687998,0.9926514877725575,0.9996055651398734,0.9996662445428656,6922,80,42,8240,0.5,0.6093423366546631,0.9930643507629214,0.9926514877725575,8105,0.9891425046267736,0.9665378162175431,0.9992783951508154,0.988860325621251,0.9940420644605555,0.8805194805194805,0.9970588235294118,0.9351724137931035,0.9532237673830595,0.9882044560943644,0.9703989703989704,6924,41,37,1,339,0,4,5,754,1103,340,0,6,757,47.8,1
+0,5071,1,15284,0.03071882672660546,0.9908400942161738,0.9876617153809296,0.995532480077276,0.9915814792543596,0.9995948737453217,0.9996413817236148,6899,103,37,8245,0.5,0.8114020824432373,0.9926426245326258,0.9915814792543596,8105,0.986428130783467,0.9688367306646221,0.999275887038378,0.9854327335047129,0.9923060329330553,0.952112676056338,0.9941176470588236,0.9726618705035971,0.8958579881656805,0.9921363040629095,0.9415422885572139,6900,15,87,1,338,1,4,2,757,1103,339,1,2,761,48.0,0
+1,6250,0,15284,0.039289169417130204,0.9914943732007327,0.9971944376677239,0.987080415358609,0.9921116504854369,0.9996785511518737,0.9997351922461065,6979,23,107,8175,0.5,0.03524836525321007,0.9940792653455774,0.9921116504854369,8105,0.9950647748303516,0.9875142202454477,0.9978553045467543,0.9967152242216509,0.9972849385538726,0.9853372434017595,0.9882352941176471,0.986784140969163,0.974025974025974,0.9829619921363041,0.9784735812133072,6979,4,19,3,336,1,12,1,750,1103,339,1,1,762,48.0,0
diff --git a/presentation/checkpoints/finetune/ft-synthetic/metrics.csv b/presentation/checkpoints/finetune/ft-synthetic/metrics.csv
new file mode 100644
index 0000000000000000000000000000000000000000..d34c714bc87bd8274eee830a25b6342fc1e74791
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-synthetic/metrics.csv
@@ -0,0 +1,317 @@
+step,epoch,loss,lr,examples_per_s,max_alloc_gb,max_reserved_gb,elapsed_s,eta_s,val_loss,val_f1,val_roc_auc,val_macro_f1_3way
+20,0,0.83201,1.0344827586206896e-06,72.3,12.957,13.084,8.9,4481,,,,
+40,0,0.72999,2.019704433497537e-06,67.9,12.957,13.084,18.3,4616,,,,
+60,0,0.59988,3.0049261083743844e-06,68.4,12.957,13.084,27.6,4644,,,,
+80,0,0.48503,3.990147783251232e-06,79.5,12.957,13.131,35.7,4488,,,,
+100,0,0.40467,4.975369458128079e-06,87.5,12.957,13.131,43.0,4318,,,,
+120,0,0.31657,5.960591133004926e-06,82.9,12.957,13.131,50.7,4235,,,,
+140,0,0.2838,6.945812807881773e-06,67.0,12.957,13.131,60.3,4305,,,,
+160,0,0.17943,7.93103448275862e-06,75.7,12.957,13.131,68.7,4287,,,,
+180,0,0.15817,8.916256157635468e-06,72.5,12.957,13.131,77.5,4292,,,,
+200,0,0.13846,9.901477832512314e-06,71.3,12.957,13.131,86.5,4301,,,,
+220,0,0.1941,1.0886699507389164e-05,84.9,12.957,13.131,94.1,4242,,,,
+240,0,0.15079,1.1871921182266011e-05,83.8,12.957,13.131,101.7,4196,,,,
+260,0,0.11472,1.2857142857142857e-05,83.7,12.957,13.131,109.3,4156,,,,
+280,0,0.24195,1.3842364532019705e-05,72.3,12.957,13.131,118.2,4163,,,,
+300,0,0.15646,1.4827586206896552e-05,59.7,12.957,13.131,128.9,4229,,,,
+320,0,0.10137,1.58128078817734e-05,94.4,12.957,13.131,135.7,4165,,,,
+340,0,0.10328,1.6798029556650246e-05,74.8,12.957,13.131,144.2,4158,,,,
+360,0,0.17159,1.7783251231527097e-05,63.9,12.957,13.131,154.3,4191,,,,
+380,0,0.10673,1.876847290640394e-05,69.7,12.957,13.131,163.4,4199,,,,
+400,0,0.12722,1.975369458128079e-05,74.6,12.957,13.131,172.0,4190,,,,
+420,0,0.18462,2.0738916256157637e-05,80.3,12.957,13.131,180.0,4166,,,,
+440,0,0.14868,2.1724137931034484e-05,74.7,12.957,13.131,188.6,4158,,,,
+460,0,0.12954,2.270935960591133e-05,70.6,12.957,13.131,197.6,4159,,,,
+480,0,0.11073,2.369458128078818e-05,67.9,12.957,13.131,207.0,4168,,,,
+500,0,0.09197,2.4679802955665024e-05,63.5,12.957,13.131,217.1,4187,,,,
+520,0,0.09838,2.566502463054187e-05,53.2,12.957,13.131,229.1,4240,,,,
+540,0,0.1301,2.6650246305418722e-05,80.9,12.957,13.131,237.0,4215,,,,
+560,0,0.1872,2.7635467980295566e-05,77.5,12.957,13.131,245.2,4195,,,,
+580,0,0.06986,2.8620689655172414e-05,100.1,12.957,13.131,251.6,4148,,,,
+600,0,0.1434,2.9605911330049262e-05,68.3,12.957,13.131,261.0,4150,,,,
+620,0,0.17168,2.9965383405014163e-05,55.5,12.957,13.131,272.5,4185,,,,
+640,0,0.14667,2.9902444141403545e-05,82.4,12.957,13.131,280.2,4161,,,,
+660,0,0.07716,2.983950487779293e-05,73.2,12.957,13.131,289.0,4152,,,,
+680,0,0.10066,2.9776565614182316e-05,69.6,12.957,13.131,298.2,4149,,,,
+700,0,0.07492,2.9713626350571698e-05,67.3,12.957,13.131,307.7,4150,,,,
+720,0,0.14172,2.9650687086961083e-05,83.0,12.957,13.131,315.4,4127,,,,
+740,0,0.11836,2.958774782335047e-05,71.4,12.957,13.131,324.4,4121,,,,
+760,0,0.10196,2.9524808559739854e-05,66.6,12.957,13.131,334.0,4123,,,,
+780,0,0.12372,2.9461869296129236e-05,68.1,12.957,13.131,343.4,4121,,,,
+800,0,0.06305,2.939893003251862e-05,73.4,12.957,13.131,352.1,4112,,,,
+820,0,0.10871,2.9335990768908003e-05,56.5,12.957,13.131,363.4,4132,,,,
+840,0,0.09817,2.9273051505297388e-05,81.4,12.957,13.131,371.3,4112,,,,
+860,0,0.09677,2.9210112241686773e-05,67.4,12.957,13.131,380.8,4110,,,,
+880,0,0.0733,2.914717297807616e-05,73.2,12.957,13.131,389.5,4100,,,,
+900,0,0.0814,2.9084233714465544e-05,59.6,12.957,13.131,400.3,4110,,,,
+920,0,0.06391,2.9021294450854926e-05,77.6,12.957,13.131,408.5,4095,,,,
+940,0,0.08487,2.8958355187244308e-05,59.7,12.957,13.131,419.3,4104,,,,
+960,0,0.08517,2.8895415923633693e-05,70.7,12.957,13.131,428.3,4097,,,,
+980,0,0.10618,2.883247666002308e-05,65.7,12.957,13.131,438.0,4095,,,,
+1000,0,0.08193,2.8769537396412464e-05,66.0,12.957,13.131,447.7,4093,,,,
+1020,0,0.06511,2.870659813280185e-05,85.3,12.957,13.131,455.3,4071,,,,
+1040,0,0.08035,2.8643658869191234e-05,91.7,12.957,13.131,462.2,4045,,,,
+1060,0,0.08145,2.8580719605580613e-05,75.1,12.957,13.131,470.7,4033,,,,
+1080,0,0.05205,2.8517780341969998e-05,82.3,12.957,13.131,478.5,4015,,,,
+1100,0,0.09047,2.8454841078359383e-05,71.2,12.957,13.131,487.5,4007,,,,
+1120,0,0.04945,2.839190181474877e-05,64.2,12.957,13.131,497.5,4007,,,,
+1140,0,0.04228,2.8328962551138154e-05,59.7,12.957,13.131,508.2,4013,,,,
+1160,0,0.08304,2.826602328752754e-05,69.9,12.957,13.131,517.4,4006,,,,
+1180,0,0.09385,2.820308402391692e-05,79.2,12.957,13.131,525.4,3991,,,,
+1200,0,0.06633,2.8140144760306303e-05,67.3,12.957,13.131,534.9,3986,,,,
+1220,0,0.05138,2.807720549669569e-05,73.2,12.957,13.131,543.7,3976,,,,
+1240,0,0.08378,2.8014266233085074e-05,63.0,12.957,13.131,553.8,3976,,,,
+1250,0,0.06329,2.7982796601279766e-05,70.6,12.957,13.131,558.4,3972,0.13254,0.95807,0.99773,0.92577
+1260,0,0.16045,2.795132696947446e-05,76.1,12.957,13.131,618.5,3966,,,,
+1280,0,0.06136,2.7888387705863844e-05,81.3,12.957,13.131,626.3,3950,,,,
+1300,0,0.06959,2.7825448442253226e-05,66.4,12.957,13.131,636.0,3946,,,,
+1320,0,0.07705,2.776250917864261e-05,73.6,12.957,13.131,644.7,3935,,,,
+1340,0,0.06575,2.7699569915031993e-05,60.3,12.957,13.131,655.3,3937,,,,
+1360,0,0.09833,2.763663065142138e-05,88.1,12.957,13.131,662.6,3917,,,,
+1380,0,0.09482,2.7573691387810764e-05,67.1,12.957,13.131,672.1,3912,,,,
+1400,0,0.08235,2.751075212420015e-05,65.2,12.957,13.131,681.9,3909,,,,
+1420,0,0.05704,2.744781286058953e-05,59.1,12.957,13.131,692.7,3912,,,,
+1440,0,0.05105,2.7384873596978917e-05,82.6,12.957,13.131,700.5,3895,,,,
+1460,0,0.13404,2.7321934333368302e-05,68.1,12.957,13.131,709.9,3889,,,,
+1480,0,0.07124,2.7258995069757684e-05,58.4,12.957,13.131,720.8,3892,,,,
+1500,0,0.05633,2.719605580614707e-05,94.4,12.957,13.131,727.6,3870,,,,
+1520,0,0.05471,2.713311654253645e-05,51.9,12.957,13.131,739.9,3880,,,,
+1540,0,0.06634,2.7070177278925836e-05,72.8,12.957,13.131,748.7,3870,,,,
+1560,0,0.08037,2.700723801531522e-05,77.8,12.957,13.131,757.0,3857,,,,
+1580,0,0.0644,2.6944298751704607e-05,68.3,12.957,13.131,766.3,3850,,,,
+1600,0,0.0666,2.6881359488093992e-05,84.4,12.957,13.131,773.9,3833,,,,
+1620,0,0.05793,2.6818420224483374e-05,65.9,12.957,13.131,783.6,3828,,,,
+1640,0,0.03927,2.6755480960872756e-05,88.8,12.957,13.131,790.8,3810,,,,
+1660,0,0.05537,2.669254169726214e-05,84.4,12.957,13.131,798.4,3794,,,,
+1680,0,0.11036,2.6629602433651527e-05,60.1,12.957,13.131,809.1,3794,,,,
+1700,0,0.05265,2.6566663170040912e-05,90.2,12.957,13.131,816.2,3775,,,,
+1720,0,0.05706,2.6503723906430297e-05,74.2,12.957,13.131,824.8,3765,,,,
+1740,0,0.06819,2.6440784642819682e-05,68.3,12.957,13.131,834.1,3758,,,,
+1760,0,0.06471,2.637784537920906e-05,90.1,12.957,13.131,841.2,3740,,,,
+1780,0,0.06069,2.6314906115598446e-05,78.2,12.957,13.131,849.4,3728,,,,
+1800,0,0.04246,2.625196685198783e-05,79.8,12.957,13.131,857.5,3715,,,,
+1820,0,0.03058,2.6189027588377217e-05,65.0,12.957,13.131,867.3,3710,,,,
+1840,0,0.05036,2.6126088324766602e-05,70.5,12.957,13.131,876.4,3702,,,,
+1860,0,0.05409,2.6063149061155987e-05,73.7,12.957,13.131,885.1,3692,,,,
+1880,0,0.08049,2.600020979754537e-05,73.0,12.957,13.131,893.8,3683,,,,
+1900,0,0.02993,2.593727053393475e-05,69.0,12.957,13.131,903.1,3675,,,,
+1920,0,0.03994,2.5874331270324137e-05,61.0,12.957,13.131,913.6,3673,,,,
+1940,0,0.15398,2.5811392006713522e-05,63.0,12.957,13.131,923.8,3669,,,,
+1960,0,0.0281,2.5748452743102907e-05,89.0,12.957,13.131,931.0,3653,,,,
+1980,0,0.0529,2.5685513479492292e-05,62.4,12.957,13.131,941.2,3650,,,,
+2000,0,0.0558,2.5622574215881674e-05,72.4,12.957,13.131,950.1,3640,,,,
+2020,0,0.09298,2.555963495227106e-05,74.2,12.957,13.131,958.7,3630,,,,
+2040,0,0.0799,2.549669568866044e-05,63.7,12.957,13.131,968.7,3625,,,,
+2060,0,0.03057,2.5433756425049827e-05,70.8,12.957,13.131,977.8,3617,,,,
+2080,0,0.06212,2.5370817161439212e-05,64.5,12.957,13.131,987.7,3612,,,,
+2100,0,0.07506,2.5307877897828597e-05,59.5,12.957,13.131,998.5,3610,,,,
+2120,0,0.03741,2.524493863421798e-05,56.5,12.957,13.131,1009.8,3609,,,,
+2140,0,0.04496,2.5181999370607365e-05,86.9,12.957,13.131,1017.1,3594,,,,
+2160,0,0.0753,2.511906010699675e-05,68.2,12.957,13.131,1026.5,3587,,,,
+2180,0,0.05823,2.5056120843386132e-05,83.3,12.957,13.131,1034.2,3573,,,,
+2200,0,0.08528,2.4993181579775517e-05,65.9,12.957,13.131,1043.9,3567,,,,
+2220,0,0.04813,2.4930242316164902e-05,52.2,12.957,13.131,1056.2,3570,,,,
+2240,0,0.04723,2.4867303052554284e-05,88.4,12.957,13.131,1063.4,3554,,,,
+2260,0,0.03841,2.480436378894367e-05,91.9,12.957,13.131,1070.4,3538,,,,
+2280,0,0.12451,2.4741424525333055e-05,66.0,12.957,13.131,1080.1,3532,,,,
+2300,0,0.05368,2.467848526172244e-05,64.3,12.957,13.131,1090.0,3526,,,,
+2320,0,0.03828,2.4615545998111822e-05,78.1,12.957,13.131,1098.2,3514,,,,
+2340,0,0.08122,2.4552606734501207e-05,63.7,12.957,13.131,1108.3,3509,,,,
+2360,0,0.04666,2.448966747089059e-05,75.1,12.957,13.131,1116.8,3498,,,,
+2380,0,0.05888,2.4426728207279975e-05,86.2,12.957,13.131,1124.2,3484,,,,
+2400,0,0.05046,2.436378894366936e-05,66.6,12.957,13.131,1133.8,3477,,,,
+2420,0,0.02317,2.4300849680058745e-05,71.5,12.957,13.131,1142.8,3468,,,,
+2440,0,0.04115,2.423791041644813e-05,56.9,12.957,13.131,1154.0,3466,,,,
+2460,0,0.02765,2.4174971152837512e-05,72.0,12.957,13.131,1162.9,3457,,,,
+2480,0,0.07228,2.4112031889226894e-05,89.3,12.957,13.131,1170.1,3442,,,,
+2500,0,0.07172,2.404909262561628e-05,60.2,12.957,13.131,1180.7,3438,0.0484,0.98663,0.99861,0.95553
+2520,0,0.07424,2.3986153362005665e-05,88.7,12.957,13.131,1241.3,3424,,,,
+2540,0,0.03973,2.392321409839505e-05,87.5,12.957,13.131,1248.6,3410,,,,
+2560,0,0.03905,2.3860274834784436e-05,71.9,12.957,13.131,1257.5,3401,,,,
+2580,0,0.04277,2.379733557117382e-05,61.8,12.957,13.131,1267.9,3396,,,,
+2600,0,0.04783,2.37343963075632e-05,60.6,12.957,13.131,1278.5,3392,,,,
+2620,0,0.05602,2.3671457043952585e-05,66.1,12.957,13.131,1288.1,3385,,,,
+2640,0,0.05284,2.360851778034197e-05,68.8,12.957,13.131,1297.5,3376,,,,
+2660,0,0.07047,2.3545578516731355e-05,99.0,12.957,13.131,1303.9,3360,,,,
+2680,0,0.06805,2.348263925312074e-05,66.6,12.957,13.131,1313.5,3353,,,,
+2700,0,0.01615,2.3419699989510126e-05,74.1,12.957,13.131,1322.2,3343,,,,
+2720,0,0.02705,2.3356760725899508e-05,87.5,12.957,13.131,1329.5,3330,,,,
+2740,0,0.07377,2.329382146228889e-05,70.7,12.957,13.131,1338.5,3321,,,,
+2760,0,0.04645,2.3230882198678275e-05,68.9,12.957,13.131,1347.8,3313,,,,
+2780,0,0.03217,2.316794293506766e-05,88.2,12.957,13.131,1355.1,3299,,,,
+2800,0,0.03426,2.3105003671457046e-05,66.5,12.957,13.131,1364.7,3292,,,,
+2820,0,0.0697,2.304206440784643e-05,63.7,12.957,13.131,1374.8,3286,,,,
+2840,0,0.05162,2.2979125144235813e-05,97.3,12.957,13.131,1381.3,3271,,,,
+2860,0,0.03941,2.2916185880625198e-05,64.5,12.957,13.131,1391.3,3264,,,,
+2880,0,0.03491,2.285324661701458e-05,63.0,12.957,13.131,1401.4,3258,,,,
+2900,0,0.03696,2.2790307353403965e-05,69.9,12.957,13.131,1410.6,3250,,,,
+2920,0,0.07309,2.272736808979335e-05,71.3,12.957,13.131,1419.5,3241,,,,
+2940,0,0.01983,2.2664428826182732e-05,74.6,12.957,13.131,1428.1,3231,,,,
+2960,0,0.05934,2.2601489562572118e-05,77.5,12.957,13.131,1436.4,3220,,,,
+2980,0,0.0402,2.2538550298961503e-05,70.7,12.957,13.131,1445.4,3211,,,,
+3000,0,0.02985,2.247561103535089e-05,63.2,12.957,13.131,1455.6,3205,,,,
+3020,0,0.03682,2.241267177174027e-05,84.5,12.957,13.131,1463.1,3193,,,,
+3040,0,0.03524,2.2349732508129656e-05,70.4,12.957,13.131,1472.2,3184,,,,
+3060,0,0.06919,2.2286793244519038e-05,91.9,12.957,13.131,1479.2,3171,,,,
+3080,0,0.04581,2.2223853980908423e-05,68.8,12.957,13.131,1488.5,3162,,,,
+3100,0,0.03714,2.2160914717297808e-05,62.3,12.957,13.131,1498.8,3156,,,,
+3120,0,0.0063,2.2097975453687193e-05,66.1,12.957,13.131,1508.5,3149,,,,
+3140,0,0.0513,2.203503619007658e-05,66.6,12.957,13.131,1518.1,3142,,,,
+3160,0,0.05117,2.197209692646596e-05,65.1,12.957,13.131,1527.9,3134,,,,
+3180,0,0.03976,2.1909157662855343e-05,90.7,12.957,13.131,1534.9,3121,,,,
+3200,0,0.06001,2.1846218399244728e-05,89.7,12.957,13.131,1542.1,3108,,,,
+3220,0,0.07842,2.1783279135634113e-05,80.0,12.957,13.131,1550.1,3097,,,,
+3240,0,0.02102,2.17203398720235e-05,66.0,12.957,13.131,1559.8,3090,,,,
+3260,0,0.0316,2.1657400608412884e-05,92.8,12.957,13.131,1566.7,3077,,,,
+3280,0,0.0584,2.159446134480227e-05,76.7,12.957,13.131,1575.0,3066,,,,
+3300,0,0.0381,2.1531522081191648e-05,71.4,12.957,13.131,1584.0,3058,,,,
+3320,0,0.0415,2.1468582817581033e-05,65.2,12.957,13.131,1593.8,3050,,,,
+3340,0,0.02009,2.1405643553970418e-05,77.7,12.957,13.131,1602.0,3040,,,,
+3360,0,0.03963,2.1342704290359803e-05,85.8,12.957,13.131,1609.5,3028,,,,
+3380,0,0.02872,2.127976502674919e-05,69.9,12.957,13.131,1618.6,3020,,,,
+3400,0,0.02821,2.1216825763138574e-05,67.9,12.957,13.131,1628.1,3012,,,,
+3420,0,0.07429,2.1153886499527956e-05,56.5,12.957,13.131,1639.4,3007,,,,
+3440,0,0.07822,2.1090947235917338e-05,74.0,12.957,13.131,1648.0,2998,,,,
+3460,0,0.02448,2.1028007972306723e-05,68.6,12.957,13.131,1657.4,2990,,,,
+3480,0,0.01664,2.096506870869611e-05,72.5,12.957,13.131,1666.2,2981,,,,
+3500,0,0.03011,2.0902129445085494e-05,76.0,12.957,13.131,1674.6,2971,,,,
+3520,0,0.04762,2.083919018147488e-05,59.1,12.957,13.131,1685.4,2965,,,,
+3540,0,0.03125,2.077625091786426e-05,66.9,12.957,13.131,1695.0,2957,,,,
+3560,0,0.04156,2.0713311654253646e-05,74.7,12.957,13.131,1703.6,2948,,,,
+3580,0,0.06968,2.0650372390643028e-05,70.4,12.957,13.131,1712.7,2939,,,,
+3600,0,0.03229,2.0587433127032413e-05,70.8,12.957,13.131,1721.7,2930,,,,
+3620,0,0.03649,2.05244938634218e-05,81.5,12.957,13.131,1729.6,2919,,,,
+3640,0,0.02746,2.0461554599811184e-05,79.5,12.957,13.131,1737.6,2909,,,,
+3660,0,0.028,2.0398615336200566e-05,75.5,12.957,13.131,1746.1,2899,,,,
+3680,0,0.11512,2.033567607258995e-05,76.8,12.957,13.131,1754.4,2889,,,,
+3700,0,0.02367,2.0272736808979337e-05,62.7,12.957,13.131,1764.6,2882,,,,
+3720,0,0.02857,2.020979754536872e-05,63.5,12.957,13.131,1774.7,2875,,,,
+3740,0,0.02283,2.0146858281758104e-05,78.9,12.957,13.131,1782.8,2865,,,,
+3750,0,0.00948,2.0115388649952796e-05,79.8,12.957,13.131,1786.8,2859,0.04945,0.98901,0.99931,0.97015
+3760,0,0.04678,2.008391901814749e-05,99.0,12.957,13.131,1843.5,2853,,,,
+3780,0,0.04648,2.002097975453687e-05,83.7,12.957,13.131,1851.2,2842,,,,
+3800,0,0.04003,1.9958040490926256e-05,92.8,12.957,13.131,1858.1,2829,,,,
+3820,0,0.03908,1.989510122731564e-05,62.4,12.957,13.131,1868.3,2823,,,,
+3840,0,0.0362,1.9832161963705027e-05,72.3,12.957,13.131,1877.2,2814,,,,
+3860,0,0.05238,1.976922270009441e-05,58.5,12.957,13.131,1888.1,2808,,,,
+3880,0,0.02373,1.9706283436483794e-05,73.9,12.957,13.131,1896.8,2799,,,,
+3900,0,0.06243,1.9643344172873176e-05,79.7,12.957,13.131,1904.8,2788,,,,
+3920,0,0.07052,1.958040490926256e-05,68.7,12.957,13.131,1914.1,2780,,,,
+3940,0,0.03893,1.9517465645651947e-05,52.8,12.957,13.131,1926.2,2776,,,,
+3960,0,0.04644,1.9454526382041332e-05,75.3,12.957,13.131,1934.7,2766,,,,
+3980,0,0.03022,1.9391587118430717e-05,73.7,12.957,13.131,1943.4,2757,,,,
+4000,0,0.01264,1.93286478548201e-05,64.3,12.957,13.131,1953.4,2750,,,,
+4020,0,0.04055,1.926570859120948e-05,81.3,12.957,13.131,1961.2,2739,,,,
+4040,0,0.02204,1.9202769327598866e-05,63.0,12.957,13.131,1971.4,2732,,,,
+4060,0,0.04293,1.913983006398825e-05,81.1,12.957,13.131,1979.3,2721,,,,
+4080,0,0.04768,1.9076890800377637e-05,69.0,12.957,13.131,1988.6,2713,,,,
+4100,0,0.02999,1.9013951536767022e-05,73.7,12.957,13.131,1997.2,2703,,,,
+4120,0,0.05806,1.8951012273156407e-05,65.3,12.957,13.131,2007.0,2696,,,,
+4140,0,0.05654,1.8888073009545786e-05,69.1,12.957,13.131,2016.3,2687,,,,
+4160,0,0.03235,1.882513374593517e-05,66.6,12.957,13.131,2025.9,2679,,,,
+4180,0,0.03999,1.8762194482324557e-05,69.3,12.957,13.131,2035.2,2671,,,,
+4200,0,0.02406,1.8699255218713942e-05,70.3,12.957,13.131,2044.3,2662,,,,
+4220,0,0.01162,1.8636315955103327e-05,71.0,12.957,13.131,2053.3,2653,,,,
+4240,0,0.00916,1.8573376691492712e-05,66.2,12.957,13.131,2062.9,2645,,,,
+4260,0,0.03192,1.8510437427882094e-05,78.9,12.957,13.131,2071.0,2635,,,,
+4280,0,0.02482,1.8447498164271476e-05,70.6,12.957,13.131,2080.1,2626,,,,
+4300,0,0.0162,1.838455890066086e-05,60.2,12.957,13.131,2090.7,2619,,,,
+4320,0,0.03451,1.8321619637050247e-05,68.3,12.957,13.131,2100.1,2611,,,,
+4340,0,0.04593,1.8258680373439632e-05,61.7,12.957,13.131,2110.5,2604,,,,
+4360,0,0.03908,1.8195741109829014e-05,63.4,12.957,13.131,2120.6,2596,,,,
+4380,0,0.03694,1.81328018462184e-05,74.7,12.957,13.131,2129.1,2587,,,,
+4400,0,0.02817,1.8069862582607785e-05,77.3,12.957,13.131,2137.4,2577,,,,
+4420,0,0.02554,1.8006923318997167e-05,73.8,12.957,13.131,2146.1,2568,,,,
+4440,0,0.01908,1.7943984055386552e-05,85.9,12.957,13.131,2153.5,2557,,,,
+4460,0,0.02582,1.7881044791775937e-05,91.7,12.957,13.131,2160.5,2545,,,,
+4480,0,0.04062,1.781810552816532e-05,81.0,12.957,13.131,2168.4,2535,,,,
+4500,0,0.03169,1.7755166264554704e-05,80.4,12.957,13.131,2176.4,2525,,,,
+4520,0,0.02649,1.769222700094409e-05,98.3,12.957,13.131,2182.9,2513,,,,
+4540,0,0.06093,1.7629287737333475e-05,56.2,12.957,13.131,2194.3,2507,,,,
+4560,0,0.04555,1.7566348473722857e-05,70.3,12.957,13.131,2203.4,2498,,,,
+4580,0,0.04437,1.7503409210112242e-05,63.3,12.957,13.131,2213.5,2490,,,,
+4600,0,0.04412,1.7440469946501624e-05,74.5,12.957,13.131,2222.1,2481,,,,
+4620,0,0.04647,1.737753068289101e-05,65.0,12.957,13.131,2231.9,2473,,,,
+4640,0,0.03226,1.7314591419280395e-05,76.8,12.957,13.131,2240.3,2464,,,,
+4660,0,0.00808,1.725165215566978e-05,73.2,12.957,13.131,2249.0,2454,,,,
+4680,0,0.02736,1.7188712892059165e-05,62.1,12.957,13.131,2259.3,2447,,,,
+4700,0,0.0475,1.7125773628448547e-05,76.5,12.957,13.131,2267.7,2437,,,,
+4720,0,0.01379,1.706283436483793e-05,75.3,12.957,13.131,2276.2,2428,,,,
+4740,0,0.04385,1.6999895101227314e-05,55.9,12.957,13.131,2287.7,2422,,,,
+4760,0,0.02186,1.69369558376167e-05,62.8,12.957,13.131,2297.8,2414,,,,
+4780,0,0.01594,1.6874016574006085e-05,64.5,12.957,13.131,2307.8,2406,,,,
+4800,0,0.02238,1.681107731039547e-05,82.3,12.957,13.131,2315.5,2396,,,,
+4820,0,0.0353,1.6748138046784856e-05,62.1,12.957,13.131,2325.8,2388,,,,
+4840,0,0.03967,1.6685198783174238e-05,67.9,12.957,13.131,2335.3,2380,,,,
+4860,0,0.0333,1.662225951956362e-05,68.3,12.957,13.131,2344.6,2371,,,,
+4880,0,0.0391,1.6559320255953005e-05,72.5,12.957,13.131,2353.5,2362,,,,
+4900,0,0.02478,1.649638099234239e-05,62.3,12.957,13.131,2363.7,2355,,,,
+4920,0,0.01545,1.6433441728731775e-05,90.5,12.957,13.131,2370.8,2344,,,,
+4940,0,0.03861,1.637050246512116e-05,69.8,12.957,13.131,2380.0,2335,,,,
+4960,0,0.05403,1.6307563201510543e-05,82.9,12.957,13.131,2387.7,2325,,,,
+4980,0,0.01699,1.6244623937899928e-05,71.1,12.957,13.131,2396.7,2316,,,,
+5000,0,0.02993,1.618168467428931e-05,76.9,12.957,13.131,2405.0,2306,0.02776,0.99265,0.99961,0.96654
+5020,0,0.02849,1.6118745410678695e-05,86.7,12.957,13.131,2465.6,2295,,,,
+5040,0,0.04676,1.605580614706808e-05,72.7,12.957,13.131,2474.4,2286,,,,
+5060,0,0.03479,1.5992866883457466e-05,74.2,12.957,13.131,2483.0,2277,,,,
+5071,0,0.04003,1.5958250288471625e-05,75.9,12.957,13.131,2487.7,2272,0.03072,0.99158,0.99959,0.96884
+5080,1,0.00384,1.5929927619846848e-05,61.6,12.957,13.131,2544.3,2268,,,,
+5100,1,0.00851,1.5866988356236233e-05,71.7,12.957,13.131,2553.2,2259,,,,
+5120,1,0.01684,1.5804049092625618e-05,64.6,12.957,13.131,2563.1,2251,,,,
+5140,1,0.01269,1.5741109829015e-05,94.0,12.957,13.131,2569.9,2240,,,,
+5160,1,0.01455,1.5678170565404385e-05,65.8,12.957,13.131,2579.7,2232,,,,
+5180,1,0.0207,1.561523130179377e-05,85.5,12.957,13.131,2587.2,2222,,,,
+5200,1,0.04711,1.5552292038183153e-05,69.7,12.957,13.131,2596.3,2213,,,,
+5220,1,0.03015,1.5489352774572538e-05,84.4,12.957,13.131,2603.9,2203,,,,
+5240,1,0.01467,1.5426413510961923e-05,61.2,12.957,13.131,2614.4,2195,,,,
+5260,1,0.00733,1.536347424735131e-05,57.9,12.957,13.131,2625.4,2188,,,,
+5280,1,0.03411,1.530053498374069e-05,81.9,12.957,13.131,2633.2,2178,,,,
+5300,1,0.03478,1.5237595720130076e-05,75.1,12.957,13.131,2641.8,2169,,,,
+5320,1,0.02079,1.5174656456519458e-05,70.9,12.957,13.131,2650.8,2160,,,,
+5340,1,0.01665,1.5111717192908843e-05,75.1,12.957,13.131,2659.3,2151,,,,
+5360,1,0.0222,1.5048777929298228e-05,79.9,12.957,13.131,2667.3,2141,,,,
+5380,1,0.02154,1.4985838665687612e-05,59.3,12.957,13.131,2678.1,2133,,,,
+5400,1,0.03667,1.4922899402076995e-05,91.7,12.957,13.131,2685.1,2123,,,,
+5420,1,0.01972,1.485996013846638e-05,86.4,12.957,13.131,2692.5,2112,,,,
+5440,1,0.00805,1.4797020874855764e-05,74.9,12.957,13.131,2701.0,2103,,,,
+5460,1,0.03173,1.4734081611245148e-05,56.0,12.957,13.131,2712.5,2096,,,,
+5480,1,0.02264,1.4671142347634533e-05,81.4,12.957,13.131,2720.3,2086,,,,
+5500,1,0.02376,1.4608203084023918e-05,65.6,12.957,13.131,2730.1,2078,,,,
+5520,1,0.00737,1.45452638204133e-05,69.0,12.957,13.131,2739.4,2069,,,,
+5540,1,0.03725,1.4482324556802686e-05,69.7,12.957,13.131,2748.5,2061,,,,
+5560,1,0.01259,1.4419385293192071e-05,63.3,12.957,13.131,2758.6,2053,,,,
+5580,1,0.02063,1.4356446029581453e-05,63.3,12.957,13.131,2768.8,2045,,,,
+5600,1,0.02369,1.4293506765970838e-05,58.3,12.957,13.131,2779.7,2037,,,,
+5620,1,0.00926,1.4230567502360223e-05,80.9,12.957,13.131,2787.7,2027,,,,
+5640,1,0.015,1.4167628238749607e-05,70.6,12.957,13.131,2796.7,2019,,,,
+5660,1,0.02596,1.410468897513899e-05,61.7,12.957,13.131,2807.1,2011,,,,
+5680,1,0.01771,1.4041749711528376e-05,87.6,12.957,13.131,2814.4,2000,,,,
+5700,1,0.00486,1.397881044791776e-05,91.7,12.957,13.131,2821.4,1990,,,,
+5720,1,0.01631,1.3915871184307143e-05,61.3,12.957,13.131,2831.8,1982,,,,
+5740,1,0.0098,1.3852931920696528e-05,99.3,12.957,13.131,2838.3,1971,,,,
+5760,1,0.01233,1.3789992657085912e-05,72.6,12.957,13.131,2847.1,1962,,,,
+5780,1,0.02041,1.3727053393475297e-05,67.9,12.957,13.131,2856.5,1954,,,,
+5800,1,0.03051,1.3664114129864681e-05,71.3,12.957,13.131,2865.5,1945,,,,
+5820,1,0.03765,1.3601174866254065e-05,81.7,12.957,13.131,2873.3,1935,,,,
+5840,1,0.02113,1.353823560264345e-05,76.1,12.957,13.131,2881.7,1925,,,,
+5860,1,0.0093,1.3475296339032835e-05,78.8,12.957,13.131,2889.9,1916,,,,
+5880,1,0.03191,1.3412357075422217e-05,72.7,12.957,13.131,2898.7,1907,,,,
+5900,1,0.01202,1.3349417811811602e-05,67.7,12.957,13.131,2908.1,1898,,,,
+5920,1,0.00833,1.3286478548200988e-05,65.8,12.957,13.131,2917.8,1890,,,,
+5940,1,0.00574,1.322353928459037e-05,82.1,12.957,13.131,2925.6,1880,,,,
+5960,1,0.03639,1.3160600020979755e-05,69.9,12.957,13.131,2934.8,1871,,,,
+5980,1,0.00908,1.309766075736914e-05,62.8,12.957,13.131,2945.0,1863,,,,
+6000,1,0.01148,1.3034721493758524e-05,65.4,12.957,13.131,2954.8,1855,,,,
+6020,1,0.01491,1.2971782230147907e-05,74.1,12.957,13.131,2963.4,1846,,,,
+6040,1,0.02988,1.2908842966537293e-05,63.6,12.957,13.131,2973.5,1837,,,,
+6060,1,0.01392,1.2845903702926676e-05,64.3,12.957,13.131,2983.4,1829,,,,
+6080,1,0.03267,1.278296443931606e-05,74.5,12.957,13.131,2992.0,1820,,,,
+6100,1,0.01692,1.2720025175705445e-05,71.6,12.957,13.131,3001.0,1811,,,,
+6120,1,0.01324,1.2657085912094829e-05,71.7,12.957,13.131,3009.9,1802,,,,
+6140,1,0.01847,1.2594146648484214e-05,77.6,12.957,13.131,3018.1,1793,,,,
+6160,1,0.0269,1.2531207384873598e-05,82.4,12.957,13.131,3025.9,1783,,,,
+6180,1,0.02294,1.2468268121262981e-05,61.2,12.957,13.131,3036.3,1775,,,,
+6200,1,0.00633,1.2405328857652367e-05,56.0,12.957,13.131,3047.8,1767,,,,
+6220,1,0.00464,1.234238959404175e-05,70.4,12.957,13.131,3056.9,1759,,,,
+6240,1,0.01013,1.2279450330431134e-05,90.3,12.957,13.131,3064.0,1748,,,,
+6250,1,0.01773,1.2247980698625826e-05,50.8,12.957,13.131,3070.3,1745,0.03929,0.99211,0.99968,0.98751
diff --git a/presentation/checkpoints/finetune/ft-weighted/epochs.csv b/presentation/checkpoints/finetune/ft-weighted/epochs.csv
new file mode 100644
index 0000000000000000000000000000000000000000..e36b5721c70747beea9b3e6c4852da9f7b4b642c
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-weighted/epochs.csv
@@ -0,0 +1,11 @@
+epoch,step,is_epoch_end,n,val_loss,accuracy,precision,recall,f1,roc_auc,pr_auc,tn,fp,fn,tp,threshold,f1_optimal_threshold,f1_at_optimal_threshold,f1_unwanted,n_3way,accuracy_3way,macro_f1_3way,precision_legitimate,recall_legitimate,f1_legitimate,precision_phishing,recall_phishing,f1_phishing,precision_spam,recall_spam,f1_spam,cm3_ll,cm3_lp,cm3_ls,cm3_pl,cm3_pp,cm3_ps,cm3_sl,cm3_sp,cm3_ss,n_unwanted_3way,pvs_phish_as_phish,pvs_phish_as_spam,pvs_spam_as_phish,pvs_spam_as_spam,eval_seconds,is_best
+0,1250,0,15284,0.07395854580623883,0.9808296257524208,0.9715499940975092,0.9937213233518474,0.9825105951172924,0.9983106561957217,0.9984907586542064,6761,241,52,8230,0.5,0.7250033020973206,0.9834452975047985,0.9825105951172924,8105,0.9680444170265269,0.9268405687066877,0.9989660265878877,0.9658668951728078,0.9821376706360732,0.8953168044077136,0.9558823529411765,0.9246088193456614,0.779835390946502,0.9934469200524246,0.8737752161383285,6763,37,202,3,325,12,4,1,758,1103,327,13,1,762,48.6,1
+0,2500,0,15284,0.048085616922478376,0.9857367181366135,0.9769339957416607,0.9972228930210094,0.9869741873804971,0.9992786498851779,0.999370426222439,6807,195,23,8259,0.5,0.9268106818199158,0.9899879372738238,0.9869741873804971,8105,0.9759407772979642,0.9321038567332335,0.998974659440457,0.9740074264495858,0.9863330681900354,0.789598108747045,0.9823529411764705,0.8754914809960681,0.8842105263157894,0.9908256880733946,0.934487021013597,6820,88,94,1,334,5,6,1,756,1103,335,5,2,761,47.7,1
+0,3750,0,15284,0.03336762119980577,0.9893352525516881,0.9927175628110207,0.98756339048539,0.9901337691423038,0.9993637585590649,0.9994219455655848,6942,60,103,8179,0.5,0.31525179743766785,0.9907105802871276,0.9901337691423038,8105,0.9914867365823565,0.978465020921087,0.9989926608145057,0.9914310197086547,0.9951974768833776,0.9548022598870056,0.9941176470588236,0.9740634005763689,0.942643391521197,0.9908256880733946,0.9661341853035144,6942,14,46,2,338,0,5,2,756,1103,340,0,3,760,47.7,1
+0,5000,0,15284,0.03646752799455797,0.9901858152316148,0.9872962607861937,0.9946872736054093,0.9909779862865391,0.9994452545762444,0.9995306862934726,6896,106,44,8238,0.5,0.7389414310455322,0.9916199433291132,0.9909779862865391,8105,0.9858112276372609,0.9644023292181444,0.9989863886475528,0.9852899171665238,0.9920908829450676,0.9234972677595629,0.9941176470588236,0.9575070821529745,0.9039615846338536,0.9868938401048493,0.943609022556391,6899,23,80,2,338,0,5,5,753,1103,339,1,6,757,47.7,1
+0,5009,1,15284,0.04922735158356969,0.9871761319026433,0.9783483199242783,0.9984303308379618,0.9882873192303095,0.9995398820401196,0.9996057637079674,6819,183,13,8269,0.5,0.9843561053276062,0.9922077922077921,0.9882873192303095,8105,0.9774213448488587,0.947350218212911,0.9998534583821805,0.9744358754641531,0.9869810501952843,0.8941798941798942,0.9941176470588236,0.9415041782729805,0.8427464008859358,0.9973787680209698,0.9135654261704682,6823,38,141,1,338,1,0,2,761,1103,339,1,2,761,47.7,0
+1,6250,0,15284,0.037448642220181405,0.9908400942161738,0.9866124790819986,0.9966191741125332,0.9915905814512254,0.9995159557337638,0.9995528514328369,6890,112,28,8254,0.5,0.9125925898551941,0.992822245008746,0.9915905814512254,8105,0.9854410857495374,0.9591417238473142,0.9995649021029732,0.9842902027992002,0.9918687486507879,0.8818897637795275,0.9882352941176471,0.9320388349514563,0.9155609167671894,0.9947575360419397,0.9535175879396985,6892,43,67,1,336,3,2,2,759,1103,337,3,2,761,47.7,1
+1,7500,0,15284,0.03039762099292758,0.9909055221146297,0.9890690690690691,0.9942042984786283,0.9916300355271873,0.999533751732437,0.9996039006872028,6911,91,48,8234,0.5,0.9067297577857971,0.9919486651734366,0.9916300355271873,8105,0.9885256014805676,0.9714799458956026,0.9992775610460916,0.9877177949157384,0.9934640522875817,0.9388888888888889,0.9941176470588236,0.9657142857142857,0.9199029126213593,0.9934469200524246,0.9552614996849401,6916,21,65,1,338,1,4,1,758,1103,339,1,2,761,48.0,1
+1,8750,0,15284,0.026731603153710964,0.9931954985605862,0.992176215695715,0.9952909925138855,0.9937311633514165,0.9997132550737048,0.9997550988092373,6937,65,39,8243,0.5,0.35187023878097534,0.9939788053949904,0.9937311633514165,8105,0.9912399753238742,0.9789970222526128,0.9989922257414339,0.9910025706940874,0.9949813593346717,0.9629629629629629,0.9941176470588236,0.9782923299565847,0.9368811881188119,0.9921363040629095,0.9637173774665818,6939,12,51,2,338,0,5,1,757,1103,340,0,2,761,47.7,1
+1,10000,0,15284,0.025379083498832725,0.9931954985605862,0.9911121787172712,0.9963776865491427,0.99373795761079,0.9997532357160727,0.9997920087331971,6928,74,30,8252,0.5,0.8991281390190125,0.9942622455758894,0.99373795761079,8105,0.9902529302899444,0.9774841428658321,0.9992789154888952,0.9895744073121965,0.9944029850746269,0.9630681818181818,0.9970588235294118,0.9797687861271677,0.9255189255189256,0.9934469200524246,0.9582806573957017,6929,12,61,1,339,0,4,1,758,1103,339,1,2,761,47.8,1
+1,10018,1,15284,0.02538144083055518,0.9931954985605862,0.9911121787172712,0.9963776865491427,0.99373795761079,0.9997535030009363,0.9997922303964539,6928,74,30,8252,0.5,0.9011234641075134,0.9943223000724812,0.99373795761079,8105,0.9902529302899444,0.9774841428658321,0.9992789154888952,0.9895744073121965,0.9944029850746269,0.9630681818181818,0.9970588235294118,0.9797687861271677,0.9255189255189256,0.9934469200524246,0.9582806573957017,6929,12,61,1,339,0,4,1,758,1103,339,1,2,761,47.8,0
diff --git a/presentation/checkpoints/finetune/ft-weighted/metrics.csv b/presentation/checkpoints/finetune/ft-weighted/metrics.csv
new file mode 100644
index 0000000000000000000000000000000000000000..38bd23ace79372da321e329b8639cacaa7f5b613
--- /dev/null
+++ b/presentation/checkpoints/finetune/ft-weighted/metrics.csv
@@ -0,0 +1,507 @@
+step,epoch,loss,lr,examples_per_s,max_alloc_gb,max_reserved_gb,elapsed_s,eta_s,val_loss,val_f1,val_roc_auc,val_macro_f1_3way
+20,0,0.49876,1.0465116279069768e-06,59.3,12.957,13.131,10.8,5392,,,,
+40,0,0.43129,2.043189368770764e-06,63.7,12.957,13.131,20.8,5198,,,,
+60,0,0.30982,3.0398671096345517e-06,82.0,12.957,13.131,28.6,4754,,,,
+80,0,0.21411,4.036544850498338e-06,92.0,12.957,13.131,35.6,4422,,,,
+100,0,0.19163,5.033222591362126e-06,80.5,12.957,13.131,43.5,4319,,,,
+120,0,0.13842,6.029900332225914e-06,85.5,12.957,13.131,51.0,4209,,,,
+140,0,0.16337,7.026578073089702e-06,72.7,12.957,13.131,59.8,4222,,,,
+160,0,0.18472,8.023255813953488e-06,85.1,12.957,13.131,67.4,4150,,,,
+180,0,0.10288,9.019933554817277e-06,66.7,12.957,13.131,77.0,4206,,,,
+200,0,0.12062,1.0016611295681064e-05,104.7,12.957,13.131,83.1,4078,,,,
+220,0,0.11586,1.1013289036544852e-05,63.3,12.957,13.131,93.2,4150,,,,
+240,0,0.14812,1.2009966777408639e-05,82.2,12.957,13.131,101.0,4114,,,,
+260,0,0.12263,1.3006644518272425e-05,70.1,12.957,13.131,110.1,4133,,,,
+280,0,0.06131,1.4003322259136214e-05,67.3,12.957,13.131,119.6,4160,,,,
+300,0,0.05974,1.5e-05,70.1,12.957,13.131,128.8,4171,,,,
+320,0,0.08916,1.5996677740863787e-05,66.3,12.957,13.131,138.4,4195,,,,
+340,0,0.08524,1.6993355481727575e-05,65.3,12.957,13.131,148.2,4219,,,,
+360,0,0.034,1.7990033222591364e-05,95.6,12.957,13.131,154.9,4156,,,,
+380,0,0.11889,1.898671096345515e-05,80.8,12.957,13.131,162.8,4130,,,,
+400,0,0.07571,1.9983388704318937e-05,69.7,12.957,13.131,172.0,4136,,,,
+420,0,0.09679,2.0980066445182726e-05,88.2,12.957,13.131,179.3,4097,,,,
+440,0,0.08475,2.197674418604651e-05,62.5,12.957,13.131,189.5,4125,,,,
+460,0,0.03318,2.29734219269103e-05,96.8,12.957,13.131,196.1,4075,,,,
+480,0,0.10643,2.3970099667774087e-05,70.9,12.957,13.131,205.1,4076,,,,
+500,0,0.05904,2.4966777408637872e-05,66.8,12.957,13.131,214.7,4087,,,,
+520,0,0.083,2.596345514950166e-05,73.7,12.957,13.133,223.4,4080,,,,
+540,0,0.05491,2.696013289036545e-05,80.9,12.957,13.133,231.3,4060,,,,
+560,0,0.06233,2.7956810631229237e-05,72.1,12.957,13.133,240.2,4057,,,,
+580,0,0.16414,2.8953488372093026e-05,68.3,12.957,13.133,249.6,4061,,,,
+600,0,0.1315,2.9950166112956814e-05,75.5,12.957,13.133,258.0,4050,,,,
+620,0,0.06006,2.9942650807136788e-05,91.0,12.957,13.133,265.1,4018,,,,
+640,0,0.04888,2.9878929481733223e-05,92.3,12.957,13.133,272.0,3986,,,,
+660,0,0.09603,2.981520815632965e-05,53.8,12.957,13.133,283.9,4025,,,,
+680,0,0.05256,2.9751486830926084e-05,72.6,12.957,13.133,292.7,4020,,,,
+700,0,0.06245,2.9687765505522515e-05,76.2,12.957,13.133,301.1,4008,,,,
+720,0,0.05529,2.9624044180118947e-05,57.2,12.957,13.133,311.9,4027,,,,
+740,0,0.10115,2.956032285471538e-05,79.1,12.957,13.133,320.0,4012,,,,
+760,0,0.09277,2.949660152931181e-05,81.3,12.957,13.133,327.8,3993,,,,
+780,0,0.04143,2.9432880203908243e-05,84.8,12.957,13.133,335.4,3972,,,,
+800,0,0.03725,2.936915887850467e-05,71.7,12.957,13.133,344.3,3967,,,,
+820,0,0.08632,2.9305437553101107e-05,67.2,12.957,13.133,353.8,3969,,,,
+840,0,0.05686,2.924171622769754e-05,66.0,12.957,13.133,363.5,3972,,,,
+860,0,0.02064,2.9177994902293967e-05,91.0,12.957,13.133,370.5,3946,,,,
+880,0,0.06453,2.91142735768904e-05,71.7,12.957,13.133,379.5,3940,,,,
+900,0,0.06165,2.905055225148683e-05,73.5,12.957,13.133,388.2,3933,,,,
+920,0,0.02502,2.8986830926083263e-05,97.7,12.957,13.133,394.7,3904,,,,
+940,0,0.06911,2.8923109600679695e-05,71.4,12.957,13.133,403.7,3899,,,,
+960,0,0.03406,2.8859388275276127e-05,80.6,12.957,13.133,411.6,3884,,,,
+980,0,0.08049,2.8795666949872555e-05,79.9,12.957,13.133,419.6,3870,,,,
+1000,0,0.07165,2.873194562446899e-05,66.3,12.957,13.133,429.3,3871,,,,
+1020,0,0.04486,2.8668224299065423e-05,71.5,12.957,13.133,438.2,3866,,,,
+1040,0,0.09806,2.860450297366185e-05,67.3,12.957,13.133,447.7,3865,,,,
+1060,0,0.04864,2.8540781648258283e-05,74.4,12.957,13.133,456.3,3857,,,,
+1080,0,0.05091,2.8477060322854715e-05,63.0,12.957,13.133,466.5,3861,,,,
+1100,0,0.02973,2.841333899745115e-05,70.8,12.957,13.133,475.6,3855,,,,
+1120,0,0.07649,2.834961767204758e-05,72.4,12.957,13.133,484.4,3848,,,,
+1140,0,0.10388,2.828589634664401e-05,74.9,12.957,13.133,492.9,3839,,,,
+1160,0,0.06972,2.8222175021240443e-05,83.5,12.957,13.133,500.6,3823,,,,
+1180,0,0.0404,2.8158453695836875e-05,86.0,12.957,13.133,508.0,3805,,,,
+1200,0,0.02981,2.8094732370433307e-05,77.0,12.957,13.133,516.4,3794,,,,
+1220,0,0.03301,2.803101104502974e-05,85.0,12.957,13.133,523.9,3778,,,,
+1240,0,0.03068,2.7967289719626167e-05,61.7,12.957,13.133,534.3,3782,,,,
+1250,0,0.04436,2.7935429056924385e-05,94.4,12.957,13.133,537.6,3771,0.07396,0.98251,0.99831,0.92684
+1260,0,0.0513,2.79035683942226e-05,95.4,12.957,13.133,595.7,3760,,,,
+1280,0,0.03644,2.7839847068819034e-05,63.2,12.957,13.133,605.8,3762,,,,
+1300,0,0.05464,2.7776125743415463e-05,74.0,12.957,13.133,614.4,3754,,,,
+1320,0,0.04292,2.7712404418011895e-05,68.2,12.957,13.133,623.8,3750,,,,
+1340,0,0.03157,2.7648683092608327e-05,79.2,12.957,13.133,631.9,3738,,,,
+1360,0,0.0563,2.758496176720476e-05,67.3,12.957,13.133,641.4,3735,,,,
+1380,0,0.08354,2.752124044180119e-05,68.8,12.957,13.133,650.7,3731,,,,
+1400,0,0.04057,2.7457519116397622e-05,65.2,12.957,13.133,660.5,3730,,,,
+1420,0,0.05655,2.739379779099405e-05,70.7,12.957,13.133,669.6,3723,,,,
+1440,0,0.04257,2.7330076465590486e-05,67.1,12.957,13.133,679.1,3720,,,,
+1460,0,0.10343,2.7266355140186918e-05,62.3,12.957,13.133,689.4,3721,,,,
+1480,0,0.03256,2.720263381478335e-05,67.1,12.957,13.133,698.9,3717,,,,
+1500,0,0.03312,2.713891248937978e-05,75.3,12.957,13.133,707.4,3707,,,,
+1520,0,0.05601,2.707519116397621e-05,74.5,12.957,13.133,716.0,3698,,,,
+1540,0,0.03214,2.7011469838572646e-05,74.7,12.957,13.133,724.6,3688,,,,
+1560,0,0.06395,2.6947748513169074e-05,81.5,12.957,13.133,732.5,3675,,,,
+1580,0,0.03246,2.6884027187765506e-05,63.4,12.957,13.133,742.6,3674,,,,
+1600,0,0.04442,2.6820305862361938e-05,73.8,12.957,13.133,751.2,3665,,,,
+1620,0,0.03229,2.675658453695837e-05,63.4,12.957,13.133,761.3,3663,,,,
+1640,0,0.02332,2.6692863211554802e-05,70.6,12.957,13.133,770.4,3656,,,,
+1660,0,0.07115,2.6629141886151234e-05,76.8,12.957,13.133,778.7,3646,,,,
+1680,0,0.04411,2.6565420560747662e-05,62.5,12.957,13.133,789.0,3644,,,,
+1700,0,0.05444,2.6501699235344094e-05,58.5,12.957,13.133,799.9,3646,,,,
+1720,0,0.04137,2.643797790994053e-05,72.0,12.957,13.133,808.8,3638,,,,
+1740,0,0.03551,2.6374256584536958e-05,69.6,12.957,13.133,818.0,3631,,,,
+1760,0,0.05214,2.631053525913339e-05,66.8,12.957,13.133,827.6,3626,,,,
+1780,0,0.03404,2.6246813933729822e-05,76.5,12.957,13.133,835.9,3616,,,,
+1800,0,0.02401,2.6183092608326254e-05,71.5,12.957,13.133,844.9,3608,,,,
+1820,0,0.02832,2.6119371282922686e-05,73.1,12.957,13.133,853.6,3599,,,,
+1840,0,0.02598,2.6055649957519118e-05,96.0,12.957,13.133,860.3,3581,,,,
+1860,0,0.01993,2.599192863211555e-05,78.5,12.957,13.133,868.5,3569,,,,
+1880,0,0.0558,2.5928207306711978e-05,58.3,12.957,13.133,879.4,3570,,,,
+1900,0,0.02356,2.5864485981308413e-05,76.3,12.957,13.133,887.8,3560,,,,
+1920,0,0.03719,2.5800764655904845e-05,67.6,12.957,13.133,897.3,3554,,,,
+1940,0,0.04273,2.5737043330501274e-05,81.3,12.957,13.133,905.2,3541,,,,
+1960,0,0.00846,2.5673322005097706e-05,90.0,12.957,13.133,912.3,3526,,,,
+1980,0,0.04962,2.560960067969414e-05,66.8,12.957,13.133,921.8,3520,,,,
+2000,0,0.04606,2.554587935429057e-05,71.0,12.957,13.133,930.9,3513,,,,
+2020,0,0.04085,2.5482158028887e-05,66.1,12.957,13.133,940.5,3508,,,,
+2040,0,0.08149,2.5418436703483433e-05,71.3,12.957,13.133,949.5,3500,,,,
+2060,0,0.01919,2.5354715378079862e-05,62.3,12.957,13.133,959.8,3497,,,,
+2080,0,0.03934,2.5290994052676297e-05,62.5,12.957,13.133,970.0,3493,,,,
+2100,0,0.03071,2.522727272727273e-05,67.1,12.957,13.133,979.6,3487,,,,
+2120,0,0.02517,2.5163551401869158e-05,62.6,12.957,13.133,989.8,3484,,,,
+2140,0,0.0516,2.509983007646559e-05,53.9,12.957,13.133,1001.7,3486,,,,
+2160,0,0.01905,2.5036108751062025e-05,68.3,12.957,13.133,1011.0,3479,,,,
+2180,0,0.02008,2.4972387425658453e-05,73.5,12.957,13.133,1019.7,3470,,,,
+2200,0,0.00895,2.4908666100254885e-05,74.9,12.957,13.133,1028.3,3460,,,,
+2220,0,0.04161,2.4844944774851317e-05,60.3,12.957,13.133,1038.9,3457,,,,
+2240,0,0.06916,2.478122344944775e-05,78.8,12.957,13.133,1047.0,3446,,,,
+2260,0,0.02999,2.471750212404418e-05,63.3,12.957,13.133,1057.1,3441,,,,
+2280,0,0.0775,2.4653780798640613e-05,61.1,12.957,13.133,1067.6,3438,,,,
+2300,0,0.03515,2.4590059473237045e-05,69.4,12.957,13.133,1076.8,3430,,,,
+2320,0,0.0187,2.4526338147833473e-05,72.3,12.957,13.133,1085.7,3421,,,,
+2340,0,0.04581,2.446261682242991e-05,81.7,12.957,13.133,1093.5,3409,,,,
+2360,0,0.05283,2.439889549702634e-05,62.3,12.957,13.133,1103.8,3404,,,,
+2380,0,0.00916,2.433517417162277e-05,73.9,12.957,13.133,1112.4,3395,,,,
+2400,0,0.03922,2.42714528462192e-05,77.2,12.957,13.133,1120.7,3384,,,,
+2420,0,0.04371,2.4207731520815636e-05,83.0,12.957,13.133,1128.4,3371,,,,
+2440,0,0.01901,2.4144010195412065e-05,81.4,12.957,13.133,1136.3,3359,,,,
+2460,0,0.06407,2.4080288870008497e-05,63.1,12.957,13.133,1146.4,3354,,,,
+2480,0,0.01849,2.401656754460493e-05,63.4,12.957,13.133,1156.5,3349,,,,
+2500,0,0.02704,2.3952846219201357e-05,66.2,12.957,13.133,1166.2,3343,0.04809,0.98697,0.99928,0.9321
+2520,0,0.05098,2.3889124893797792e-05,70.0,12.957,13.133,1228.6,3334,,,,
+2540,0,0.02379,2.3825403568394224e-05,64.7,12.957,13.133,1238.5,3328,,,,
+2560,0,0.02746,2.3761682242990653e-05,65.7,12.957,13.133,1248.3,3322,,,,
+2580,0,0.01186,2.3697960917587085e-05,83.6,12.957,13.133,1255.9,3309,,,,
+2600,0,0.03863,2.363423959218352e-05,67.9,12.957,13.133,1265.4,3302,,,,
+2620,0,0.03149,2.3570518266779952e-05,64.9,12.957,13.133,1275.2,3296,,,,
+2640,0,0.02588,2.350679694137638e-05,78.0,12.957,13.133,1283.4,3285,,,,
+2660,0,0.04187,2.3443075615972812e-05,88.8,12.957,13.133,1290.6,3271,,,,
+2680,0,0.02531,2.3379354290569244e-05,73.2,12.957,13.133,1299.4,3262,,,,
+2700,0,0.04554,2.3315632965165676e-05,64.1,12.957,13.133,1309.4,3256,,,,
+2720,0,0.02802,2.3251911639762108e-05,78.4,12.957,13.133,1317.5,3245,,,,
+2740,0,0.0263,2.318819031435854e-05,66.1,12.957,13.133,1327.2,3238,,,,
+2760,0,0.04912,2.312446898895497e-05,68.7,12.957,13.133,1336.5,3231,,,,
+2780,0,0.04172,2.3060747663551404e-05,72.4,12.957,13.133,1345.4,3222,,,,
+2800,0,0.04531,2.2997026338147836e-05,60.2,12.957,13.133,1356.0,3217,,,,
+2820,0,0.02002,2.2933305012744264e-05,85.2,12.957,13.133,1363.5,3205,,,,
+2840,0,0.02024,2.2869583687340696e-05,59.3,12.957,13.133,1374.3,3201,,,,
+2860,0,0.03579,2.2805862361937128e-05,61.3,12.957,13.133,1384.7,3195,,,,
+2880,0,0.01067,2.274214103653356e-05,75.0,12.957,13.133,1393.2,3185,,,,
+2900,0,0.00566,2.2678419711129992e-05,77.9,12.957,13.133,1401.5,3175,,,,
+2920,0,0.01676,2.2614698385726424e-05,77.9,12.957,13.133,1409.7,3164,,,,
+2940,0,0.01836,2.2550977060322852e-05,61.0,12.957,13.133,1420.2,3159,,,,
+2960,0,0.04463,2.2487255734919288e-05,77.0,12.957,13.133,1428.5,3149,,,,
+2980,0,0.00934,2.242353440951572e-05,92.9,12.957,13.133,1435.4,3135,,,,
+3000,0,0.00913,2.235981308411215e-05,67.1,12.957,13.133,1444.9,3128,,,,
+3020,0,0.01563,2.229609175870858e-05,87.7,12.957,13.133,1452.2,3115,,,,
+3040,0,0.11393,2.2232370433305012e-05,76.6,12.957,13.133,1460.6,3105,,,,
+3060,0,0.02641,2.2168649107901447e-05,80.0,12.957,13.133,1468.6,3094,,,,
+3080,0,0.04185,2.2104927782497876e-05,66.8,12.957,13.133,1478.2,3086,,,,
+3100,0,0.02652,2.2041206457094308e-05,65.6,12.957,13.133,1487.9,3079,,,,
+3120,0,0.02007,2.197748513169074e-05,70.8,12.957,13.133,1496.9,3071,,,,
+3140,0,0.06407,2.191376380628717e-05,69.7,12.957,13.133,1506.1,3063,,,,
+3160,0,0.02179,2.1850042480883604e-05,81.5,12.957,13.133,1514.0,3051,,,,
+3180,0,0.02911,2.1786321155480035e-05,70.5,12.957,13.133,1523.1,3043,,,,
+3200,0,0.0196,2.1722599830076464e-05,68.7,12.957,13.133,1532.4,3035,,,,
+3220,0,0.04657,2.16588785046729e-05,73.3,12.957,13.133,1541.1,3026,,,,
+3240,0,0.01124,2.159515717926933e-05,67.6,12.957,13.133,1550.6,3018,,,,
+3260,0,0.06102,2.153143585386576e-05,69.0,12.957,13.133,1559.9,3010,,,,
+3280,0,0.01526,2.146771452846219e-05,65.1,12.957,13.133,1569.7,3003,,,,
+3300,0,0.02735,2.1403993203058624e-05,84.9,12.957,13.133,1577.2,2991,,,,
+3320,0,0.04334,2.1340271877655055e-05,72.8,12.957,13.133,1586.0,2982,,,,
+3340,0,0.01873,2.1276550552251487e-05,100.2,12.957,13.133,1592.4,2968,,,,
+3360,0,0.01176,2.121282922684792e-05,66.3,12.957,13.133,1602.0,2961,,,,
+3380,0,0.01898,2.1149107901444348e-05,67.1,12.957,13.133,1611.6,2953,,,,
+3400,0,0.01864,2.1085386576040783e-05,79.5,12.957,13.133,1619.6,2942,,,,
+3420,0,0.05191,2.1021665250637215e-05,72.7,12.957,13.133,1628.4,2933,,,,
+3440,0,0.04072,2.0957943925233647e-05,68.2,12.957,13.133,1637.8,2925,,,,
+3460,0,0.02086,2.0894222599830075e-05,73.0,12.957,13.133,1646.6,2916,,,,
+3480,0,0.04367,2.0830501274426507e-05,57.4,12.957,13.133,1657.7,2912,,,,
+3500,0,0.02283,2.0766779949022943e-05,85.1,12.957,13.133,1665.3,2900,,,,
+3520,0,0.02183,2.070305862361937e-05,95.3,12.957,13.133,1672.0,2887,,,,
+3540,0,0.0126,2.0639337298215803e-05,76.9,12.957,13.133,1680.3,2877,,,,
+3560,0,0.01476,2.0575615972812235e-05,79.4,12.957,13.133,1688.3,2867,,,,
+3580,0,0.06876,2.0511894647408667e-05,74.6,12.957,13.133,1696.9,2857,,,,
+3600,0,0.0431,2.04481733220051e-05,66.3,12.957,13.133,1706.6,2850,,,,
+3620,0,0.0211,2.038445199660153e-05,69.3,12.957,13.133,1715.8,2842,,,,
+3640,0,0.038,2.032073067119796e-05,95.4,12.957,13.133,1722.5,2829,,,,
+3660,0,0.03108,2.025700934579439e-05,75.6,12.957,13.133,1731.0,2819,,,,
+3680,0,0.0366,2.0193288020390827e-05,69.1,12.957,13.133,1740.2,2811,,,,
+3700,0,0.01227,2.0129566694987255e-05,85.7,12.957,13.133,1747.7,2800,,,,
+3720,0,0.00684,2.0065845369583687e-05,69.3,12.957,13.133,1757.0,2792,,,,
+3740,0,0.02043,2.000212404418012e-05,67.4,12.957,13.133,1766.5,2784,,,,
+3750,0,0.03868,1.9970263381478337e-05,75.5,12.957,13.133,1770.7,2779,0.03337,0.99013,0.99936,0.97847
+3760,0,0.00887,1.993840271877655e-05,73.7,12.957,13.133,1828.0,2775,,,,
+3780,0,0.00962,1.9874681393372983e-05,77.5,12.957,13.133,1836.2,2765,,,,
+3800,0,0.06242,1.9810960067969415e-05,73.4,12.957,13.133,1845.0,2756,,,,
+3820,0,0.04529,1.9747238742565847e-05,70.4,12.957,13.133,1854.0,2747,,,,
+3840,0,0.01859,1.9683517417162275e-05,78.5,12.957,13.133,1862.2,2737,,,,
+3860,0,0.04104,1.961979609175871e-05,68.5,12.957,13.133,1871.5,2729,,,,
+3880,0,0.02367,1.9556074766355142e-05,76.0,12.957,13.133,1880.0,2719,,,,
+3900,0,0.01289,1.949235344095157e-05,71.9,12.957,13.133,1888.9,2711,,,,
+3920,0,0.05216,1.9428632115548003e-05,71.3,12.957,13.133,1897.8,2702,,,,
+3940,0,0.08122,1.9364910790144438e-05,61.7,12.957,13.133,1908.2,2695,,,,
+3960,0,0.03854,1.9301189464740867e-05,76.6,12.957,13.133,1916.6,2686,,,,
+3980,0,0.04819,1.92374681393373e-05,84.1,12.957,13.133,1924.2,2675,,,,
+4000,0,0.01344,1.917374681393373e-05,68.9,12.957,13.133,1933.5,2667,,,,
+4020,0,0.01473,1.9110025488530162e-05,71.4,12.957,13.133,1942.4,2658,,,,
+4040,0,0.04244,1.9046304163126594e-05,64.3,12.957,13.133,1952.4,2651,,,,
+4060,0,0.01711,1.8982582837723026e-05,75.4,12.957,13.133,1960.9,2641,,,,
+4080,0,0.06447,1.8918861512319455e-05,59.0,12.957,13.133,1971.7,2635,,,,
+4100,0,0.01829,1.8855140186915887e-05,73.2,12.957,13.133,1980.5,2626,,,,
+4120,0,0.01321,1.8791418861512322e-05,74.0,12.957,13.133,1989.1,2617,,,,
+4140,0,0.03487,1.872769753610875e-05,65.3,12.957,13.133,1998.9,2610,,,,
+4160,0,0.04292,1.8663976210705182e-05,70.0,12.957,13.133,2008.1,2601,,,,
+4180,0,0.01251,1.8600254885301614e-05,64.8,12.957,13.133,2017.9,2594,,,,
+4200,0,0.0263,1.853653355989805e-05,73.5,12.957,13.133,2026.6,2584,,,,
+4220,0,0.01061,1.8472812234494478e-05,95.5,12.957,13.133,2033.3,2573,,,,
+4240,0,0.0288,1.840909090909091e-05,89.5,12.957,13.133,2040.5,2561,,,,
+4260,0,0.01192,1.8345369583687342e-05,58.7,12.957,13.133,2051.4,2555,,,,
+4280,0,0.01148,1.828164825828377e-05,76.3,12.957,13.133,2059.8,2546,,,,
+4300,0,0.02989,1.8217926932880206e-05,59.7,12.957,13.133,2070.5,2539,,,,
+4320,0,0.07932,1.8154205607476638e-05,80.7,12.957,13.133,2078.4,2529,,,,
+4340,0,0.02433,1.8090484282073066e-05,63.1,12.957,13.133,2088.6,2522,,,,
+4360,0,0.0157,1.8026762956669498e-05,61.4,12.957,13.133,2099.0,2515,,,,
+4380,0,0.00466,1.7963041631265933e-05,64.8,12.957,13.133,2108.9,2507,,,,
+4400,0,0.04924,1.7899320305862362e-05,61.8,12.957,13.133,2119.2,2500,,,,
+4420,0,0.03062,1.7835598980458794e-05,68.5,12.957,13.133,2128.6,2492,,,,
+4440,0,0.03051,1.7771877655055226e-05,81.7,12.957,13.133,2136.4,2482,,,,
+4460,0,0.01075,1.7708156329651654e-05,70.5,12.957,13.133,2145.5,2473,,,,
+4480,0,0.02346,1.764443500424809e-05,64.5,12.957,13.133,2155.4,2465,,,,
+4500,0,0.01461,1.758071367884452e-05,74.8,12.957,13.133,2164.0,2456,,,,
+4520,0,0.03334,1.751699235344095e-05,67.8,12.957,13.133,2173.4,2448,,,,
+4540,0,0.0061,1.7453271028037382e-05,78.0,12.957,13.133,2181.6,2438,,,,
+4560,0,0.01777,1.7389549702633817e-05,60.3,12.957,13.133,2192.2,2431,,,,
+4580,0,0.02076,1.732582837723025e-05,62.8,12.957,13.133,2202.4,2424,,,,
+4600,0,0.01929,1.7262107051826678e-05,61.9,12.957,13.133,2212.7,2417,,,,
+4620,0,0.01835,1.719838572642311e-05,67.1,12.957,13.133,2222.3,2408,,,,
+4640,0,0.0071,1.713466440101954e-05,97.5,12.957,13.133,2228.9,2397,,,,
+4660,0,0.05291,1.7070943075615973e-05,64.0,12.957,13.133,2238.9,2389,,,,
+4680,0,0.01027,1.7007221750212405e-05,75.4,12.957,13.133,2247.3,2380,,,,
+4700,0,0.00992,1.6943500424808837e-05,79.6,12.957,13.133,2255.4,2370,,,,
+4720,0,0.02822,1.6879779099405266e-05,62.8,12.957,13.133,2265.6,2362,,,,
+4740,0,0.00412,1.68160577740017e-05,70.4,12.957,13.133,2274.7,2354,,,,
+4760,0,0.04224,1.6752336448598133e-05,69.6,12.957,13.133,2283.9,2345,,,,
+4780,0,0.02594,1.668861512319456e-05,83.5,12.957,13.133,2291.5,2335,,,,
+4800,0,0.00465,1.6624893797790993e-05,71.7,12.957,13.133,2300.5,2326,,,,
+4820,0,0.01423,1.6561172472387425e-05,70.9,12.957,13.133,2309.5,2317,,,,
+4840,0,0.00876,1.6497451146983857e-05,57.5,12.957,13.133,2320.6,2310,,,,
+4860,0,0.00508,1.643372982158029e-05,82.7,12.957,13.133,2328.3,2300,,,,
+4880,0,0.01538,1.637000849617672e-05,73.1,12.957,13.133,2337.1,2291,,,,
+4900,0,0.00833,1.630628717077315e-05,61.6,12.957,13.133,2347.5,2284,,,,
+4920,0,0.00896,1.6242565845369585e-05,66.4,12.957,13.133,2357.1,2276,,,,
+4940,0,0.01066,1.6178844519966017e-05,71.4,12.957,13.133,2366.1,2267,,,,
+4960,0,0.01033,1.611512319456245e-05,62.3,12.957,13.133,2376.4,2259,,,,
+4980,0,0.04955,1.6051401869158877e-05,64.5,12.957,13.133,2386.3,2251,,,,
+5000,0,0.04992,1.5987680543755312e-05,71.2,12.957,13.133,2395.3,2242,0.03647,0.99098,0.99945,0.9644
+5009,0,0.08897,1.5959005947323706e-05,98.3,12.957,13.133,2451.1,2237,0.04923,0.98829,0.99954,0.94735
+5020,1,0.01949,1.5923959218351744e-05,81.2,12.957,13.133,2507.3,2232,,,,
+5040,1,0.01687,1.5860237892948173e-05,74.3,12.957,13.133,2515.9,2223,,,,
+5060,1,0.01589,1.5796516567544605e-05,73.1,12.957,13.133,2524.7,2213,,,,
+5080,1,0.00686,1.5732795242141037e-05,63.3,12.957,13.133,2534.8,2206,,,,
+5100,1,0.02802,1.566907391673747e-05,86.1,12.957,13.133,2542.2,2195,,,,
+5120,1,0.01098,1.56053525913339e-05,75.4,12.957,13.133,2550.7,2186,,,,
+5140,1,0.00932,1.5541631265930333e-05,66.9,12.957,13.133,2560.3,2178,,,,
+5160,1,0.00375,1.547790994052676e-05,81.4,12.957,13.133,2568.2,2168,,,,
+5180,1,0.01019,1.5414188615123196e-05,107.9,12.957,13.133,2574.1,2156,,,,
+5200,1,0.0068,1.5350467289719628e-05,68.1,12.957,13.133,2583.5,2147,,,,
+5220,1,0.00079,1.5286745964316057e-05,81.0,12.957,13.133,2591.4,2138,,,,
+5240,1,0.02176,1.5223024638912489e-05,61.9,12.957,13.133,2601.7,2130,,,,
+5260,1,0.0089,1.5159303313508922e-05,74.3,12.957,13.133,2610.3,2121,,,,
+5280,1,0.00602,1.509558198810535e-05,72.2,12.957,13.133,2619.2,2112,,,,
+5300,1,0.00882,1.5031860662701784e-05,61.0,12.957,13.133,2629.7,2104,,,,
+5320,1,0.008,1.4968139337298216e-05,57.6,12.957,13.133,2640.8,2097,,,,
+5340,1,0.05298,1.4904418011894648e-05,71.1,12.957,13.133,2649.8,2089,,,,
+5360,1,0.00129,1.4840696686491078e-05,71.5,12.957,13.133,2658.7,2080,,,,
+5380,1,0.01535,1.4776975361087512e-05,62.4,12.957,13.133,2669.0,2072,,,,
+5400,1,0.01746,1.4713254035683942e-05,62.5,12.957,13.133,2679.2,2064,,,,
+5420,1,0.03648,1.4649532710280374e-05,67.1,12.957,13.133,2688.8,2056,,,,
+5440,1,0.02154,1.4585811384876806e-05,67.0,12.957,13.133,2698.3,2047,,,,
+5460,1,0.01914,1.4522090059473236e-05,67.5,12.957,13.133,2707.8,2039,,,,
+5480,1,0.0064,1.445836873406967e-05,67.2,12.957,13.133,2717.3,2030,,,,
+5500,1,0.00937,1.43946474086661e-05,73.4,12.957,13.133,2726.1,2021,,,,
+5520,1,0.00792,1.4330926083262532e-05,85.4,12.957,13.133,2733.6,2011,,,,
+5540,1,0.00594,1.4267204757858964e-05,60.9,12.957,13.133,2744.1,2003,,,,
+5560,1,0.00594,1.4203483432455396e-05,79.1,12.957,13.133,2752.2,1994,,,,
+5580,1,0.00647,1.4139762107051826e-05,126.5,12.957,13.133,2757.2,1982,,,,
+5600,1,0.02291,1.407604078164826e-05,83.1,12.957,13.133,2764.9,1972,,,,
+5620,1,0.01117,1.401231945624469e-05,70.9,12.957,13.133,2773.9,1963,,,,
+5640,1,0.0058,1.3948598130841122e-05,73.8,12.957,13.133,2782.6,1954,,,,
+5660,1,0.01963,1.3884876805437554e-05,64.5,12.957,13.133,2792.5,1946,,,,
+5680,1,0.00715,1.3821155480033984e-05,73.4,12.957,13.133,2801.3,1936,,,,
+5700,1,0.02993,1.3757434154630418e-05,68.2,12.957,13.133,2810.6,1928,,,,
+5720,1,0.00578,1.3693712829226848e-05,78.6,12.957,13.133,2818.8,1918,,,,
+5740,1,0.00694,1.362999150382328e-05,88.9,12.957,13.133,2826.0,1908,,,,
+5760,1,0.00355,1.3566270178419712e-05,64.3,12.957,13.133,2835.9,1900,,,,
+5780,1,0.03005,1.3502548853016144e-05,65.0,12.957,13.133,2845.8,1892,,,,
+5800,1,0.03324,1.3438827527612575e-05,78.6,12.957,13.133,2853.9,1882,,,,
+5820,1,0.00329,1.3375106202209006e-05,60.2,12.957,13.133,2864.5,1875,,,,
+5840,1,0.01685,1.3311384876805438e-05,70.1,12.957,13.133,2873.7,1866,,,,
+5860,1,0.00354,1.324766355140187e-05,70.9,12.957,13.133,2882.7,1857,,,,
+5880,1,0.032,1.3183942225998301e-05,62.2,12.957,13.133,2893.0,1849,,,,
+5900,1,0.0053,1.3120220900594732e-05,70.3,12.957,13.133,2902.1,1840,,,,
+5920,1,0.00666,1.3056499575191165e-05,66.1,12.957,13.133,2911.8,1832,,,,
+5940,1,0.00685,1.2992778249787595e-05,65.6,12.957,13.133,2921.5,1823,,,,
+5960,1,0.05389,1.2929056924384027e-05,64.0,12.957,13.133,2931.5,1815,,,,
+5980,1,0.02529,1.286533559898046e-05,65.7,12.957,13.133,2941.3,1807,,,,
+6000,1,0.01949,1.2801614273576891e-05,61.5,12.957,13.133,2951.7,1799,,,,
+6020,1,0.00617,1.2737892948173323e-05,75.7,12.957,13.133,2960.1,1789,,,,
+6040,1,0.00264,1.2674171622769753e-05,75.4,12.957,13.133,2968.6,1780,,,,
+6060,1,0.00393,1.2610450297366185e-05,62.8,12.957,13.133,2978.8,1772,,,,
+6080,1,0.01276,1.2546728971962617e-05,80.0,12.957,13.133,2986.8,1762,,,,
+6100,1,0.01815,1.2483007646559049e-05,65.6,12.957,13.133,2996.6,1754,,,,
+6120,1,0.00146,1.241928632115548e-05,72.4,12.957,13.133,3005.4,1745,,,,
+6140,1,0.00719,1.2355564995751913e-05,92.9,12.957,13.133,3012.3,1735,,,,
+6160,1,0.00106,1.2291843670348343e-05,65.5,12.957,13.133,3022.1,1726,,,,
+6180,1,0.00735,1.2228122344944777e-05,69.7,12.957,13.133,3031.3,1717,,,,
+6200,1,0.00381,1.2164401019541207e-05,69.1,12.957,13.133,3040.5,1709,,,,
+6220,1,0.0048,1.2100679694137637e-05,78.5,12.957,13.133,3048.7,1699,,,,
+6240,1,0.0103,1.203695836873407e-05,80.3,12.957,13.133,3056.6,1690,,,,
+6250,1,0.04387,1.2005097706032285e-05,47.9,12.957,13.133,3063.3,1687,0.03745,0.99159,0.99952,0.95914
+6260,1,0.00913,1.1973237043330501e-05,73.4,12.957,13.133,3121.2,1682,,,,
+6280,1,0.00194,1.1909515717926933e-05,104.8,12.957,13.133,3127.3,1671,,,,
+6300,1,0.05131,1.1845794392523365e-05,66.5,12.957,13.133,3136.9,1663,,,,
+6320,1,0.00483,1.1782073067119797e-05,63.1,12.957,13.133,3147.1,1655,,,,
+6340,1,0.00169,1.1718351741716227e-05,78.5,12.957,13.133,3155.2,1645,,,,
+6360,1,0.02917,1.165463041631266e-05,114.0,12.957,13.133,3160.8,1634,,,,
+6380,1,0.00571,1.159090909090909e-05,69.9,12.957,13.133,3170.0,1626,,,,
+6400,1,0.01283,1.1527187765505523e-05,89.2,12.957,13.133,3177.2,1616,,,,
+6420,1,0.01319,1.1463466440101955e-05,66.8,12.957,13.133,3186.8,1607,,,,
+6440,1,0.03805,1.1399745114698385e-05,63.7,12.957,13.133,3196.8,1599,,,,
+6460,1,0.00186,1.1336023789294818e-05,74.3,12.957,13.133,3205.4,1590,,,,
+6480,1,0.00674,1.1272302463891249e-05,71.5,12.957,13.133,3214.4,1581,,,,
+6500,1,0.02089,1.120858113848768e-05,87.2,12.957,13.133,3221.7,1571,,,,
+6520,1,0.005,1.1144859813084113e-05,76.8,12.957,13.133,3230.0,1562,,,,
+6540,1,0.01006,1.1081138487680544e-05,73.9,12.957,13.133,3238.7,1553,,,,
+6560,1,0.00488,1.1017417162276975e-05,83.1,12.957,13.133,3246.4,1543,,,,
+6580,1,0.00383,1.0953695836873408e-05,69.0,12.957,13.133,3255.7,1534,,,,
+6600,1,0.0036,1.0889974511469838e-05,71.6,12.957,13.133,3264.6,1525,,,,
+6620,1,0.02054,1.082625318606627e-05,59.1,12.957,13.133,3275.5,1517,,,,
+6640,1,0.00211,1.0762531860662702e-05,57.5,12.957,13.133,3286.6,1510,,,,
+6660,1,0.00364,1.0698810535259133e-05,93.7,12.957,13.133,3293.4,1500,,,,
+6680,1,0.00616,1.0635089209855566e-05,75.5,12.957,13.133,3301.9,1490,,,,
+6700,1,0.00403,1.0571367884451996e-05,69.1,12.957,13.133,3311.1,1482,,,,
+6720,1,0.01653,1.0507646559048428e-05,64.3,12.957,13.133,3321.1,1473,,,,
+6740,1,0.00391,1.044392523364486e-05,68.1,12.957,13.133,3330.5,1465,,,,
+6760,1,0.00567,1.0380203908241292e-05,63.1,12.957,13.133,3340.6,1456,,,,
+6780,1,0.00171,1.0316482582837724e-05,64.7,12.957,13.133,3350.5,1448,,,,
+6800,1,0.00136,1.0252761257434154e-05,78.2,12.957,13.133,3358.7,1438,,,,
+6820,1,0.00398,1.0189039932030586e-05,59.9,12.957,13.133,3369.4,1430,,,,
+6840,1,0.00187,1.0125318606627018e-05,86.0,12.957,13.133,3376.8,1421,,,,
+6860,1,0.00382,1.006159728122345e-05,70.3,12.957,13.133,3385.9,1412,,,,
+6880,1,0.00359,9.99787595581988e-06,83.0,12.957,13.133,3393.7,1402,,,,
+6900,1,0.00313,9.934154630416314e-06,82.9,12.957,13.133,3401.4,1393,,,,
+6920,1,0.01262,9.870433305012744e-06,64.9,12.957,13.133,3411.2,1384,,,,
+6940,1,0.01651,9.806711979609176e-06,61.2,12.957,13.133,3421.7,1376,,,,
+6960,1,0.02204,9.742990654205608e-06,85.9,12.957,13.133,3429.1,1366,,,,
+6980,1,0.00234,9.67926932880204e-06,84.8,12.957,13.133,3436.7,1357,,,,
+7000,1,0.00764,9.615548003398472e-06,66.2,12.957,13.133,3446.4,1348,,,,
+7020,1,0.0377,9.551826677994902e-06,84.4,12.957,13.133,3453.9,1339,,,,
+7040,1,0.00313,9.488105352591334e-06,78.3,12.957,13.133,3462.1,1329,,,,
+7060,1,0.03583,9.424384027187766e-06,64.8,12.957,13.133,3472.0,1321,,,,
+7080,1,0.01861,9.360662701784198e-06,74.2,12.957,13.133,3480.6,1312,,,,
+7100,1,0.009,9.296941376380628e-06,69.2,12.957,13.133,3489.9,1303,,,,
+7120,1,0.00265,9.233220050977061e-06,73.5,12.957,13.133,3498.6,1294,,,,
+7140,1,0.00337,9.169498725573492e-06,68.3,12.957,13.133,3507.9,1285,,,,
+7160,1,0.00406,9.105777400169925e-06,57.8,12.957,13.133,3519.0,1277,,,,
+7180,1,0.00741,9.042056074766356e-06,83.8,12.957,13.133,3526.7,1268,,,,
+7200,1,0.02438,8.978334749362786e-06,58.2,12.957,13.133,3537.6,1260,,,,
+7220,1,0.01584,8.91461342395922e-06,78.1,12.957,13.133,3545.8,1250,,,,
+7240,1,0.00184,8.85089209855565e-06,103.1,12.957,13.133,3552.0,1240,,,,
+7260,1,0.00856,8.787170773152081e-06,57.3,12.957,13.133,3563.2,1232,,,,
+7280,1,0.00175,8.723449447748513e-06,63.6,12.957,13.133,3573.3,1224,,,,
+7300,1,0.00051,8.659728122344945e-06,59.4,12.957,13.133,3584.1,1216,,,,
+7320,1,0.00547,8.596006796941376e-06,87.5,12.957,13.133,3591.4,1206,,,,
+7340,1,0.00127,8.532285471537809e-06,83.3,12.957,13.133,3599.0,1197,,,,
+7360,1,0.003,8.46856414613424e-06,74.5,12.957,13.133,3607.6,1188,,,,
+7380,1,0.00925,8.404842820730673e-06,84.8,12.957,13.133,3615.2,1178,,,,
+7400,1,0.00273,8.341121495327103e-06,69.5,12.957,13.133,3624.4,1169,,,,
+7420,1,0.00588,8.277400169923533e-06,70.4,12.957,13.133,3633.5,1160,,,,
+7440,1,0.0094,8.213678844519967e-06,66.0,12.957,13.133,3643.2,1152,,,,
+7460,1,0.01423,8.149957519116397e-06,68.5,12.957,13.133,3652.5,1143,,,,
+7480,1,0.03058,8.086236193712829e-06,89.4,12.957,13.133,3659.7,1133,,,,
+7500,1,0.02703,8.022514868309261e-06,75.5,12.957,13.133,3668.2,1124,0.0304,0.99163,0.99953,0.97148
+7520,1,0.00902,7.958793542905693e-06,61.2,12.957,13.133,3731.5,1116,,,,
+7540,1,0.00172,7.895072217502125e-06,69.5,12.957,13.133,3740.7,1107,,,,
+7560,1,0.01355,7.831350892098557e-06,74.3,12.957,13.133,3749.3,1098,,,,
+7580,1,0.00843,7.767629566694987e-06,61.1,12.957,13.133,3759.8,1090,,,,
+7600,1,0.0139,7.703908241291419e-06,77.3,12.957,13.133,3768.1,1080,,,,
+7620,1,0.0105,7.64018691588785e-06,89.8,12.957,13.133,3775.2,1071,,,,
+7640,1,0.00201,7.576465590484282e-06,79.1,12.957,13.133,3783.3,1062,,,,
+7660,1,0.00607,7.512744265080715e-06,64.5,12.957,13.133,3793.2,1053,,,,
+7680,1,0.00087,7.449022939677146e-06,56.9,12.957,13.133,3804.5,1045,,,,
+7700,1,0.00032,7.385301614273577e-06,66.0,12.957,13.133,3814.2,1036,,,,
+7720,1,0.03329,7.321580288870009e-06,68.7,12.957,13.133,3823.5,1027,,,,
+7740,1,0.00404,7.25785896346644e-06,69.1,12.957,13.133,3832.7,1019,,,,
+7760,1,0.0112,7.194137638062872e-06,88.4,12.957,13.133,3840.0,1009,,,,
+7780,1,0.00207,7.130416312659304e-06,72.5,12.957,13.133,3848.8,1000,,,,
+7800,1,0.01141,7.0666949872557355e-06,70.3,12.957,13.133,3857.9,991,,,,
+7820,1,0.00419,7.0029736618521666e-06,70.3,12.957,13.133,3867.0,982,,,,
+7840,1,0.0059,6.9392523364485985e-06,76.6,12.957,13.133,3875.4,973,,,,
+7860,1,0.00107,6.87553101104503e-06,72.7,12.957,13.133,3884.2,964,,,,
+7880,1,0.03063,6.811809685641462e-06,64.3,12.957,13.133,3894.1,956,,,,
+7900,1,0.00856,6.7480883602378925e-06,65.5,12.957,13.133,3903.9,947,,,,
+7920,1,0.01608,6.6843670348343244e-06,60.4,12.957,13.133,3914.5,938,,,,
+7940,1,0.00149,6.620645709430756e-06,68.5,12.957,13.133,3923.8,930,,,,
+7960,1,0.00164,6.5569243840271874e-06,92.8,12.957,13.133,3930.7,920,,,,
+7980,1,0.05799,6.493203058623619e-06,68.1,12.957,13.133,3940.1,911,,,,
+8000,1,0.01437,6.429481733220051e-06,74.5,12.957,13.133,3948.7,902,,,,
+8020,1,0.02092,6.365760407816483e-06,68.3,12.957,13.133,3958.1,893,,,,
+8040,1,0.00186,6.302039082412914e-06,69.9,12.957,13.133,3967.2,884,,,,
+8060,1,0.01241,6.238317757009346e-06,68.6,12.957,13.133,3976.5,876,,,,
+8080,1,0.0109,6.174596431605777e-06,65.5,12.957,13.133,3986.3,867,,,,
+8100,1,0.01091,6.110875106202209e-06,58.1,12.957,13.133,3997.3,858,,,,
+8120,1,0.01851,6.04715378079864e-06,64.2,12.957,13.133,4007.3,850,,,,
+8140,1,0.01167,5.983432455395072e-06,69.0,12.957,13.133,4016.6,841,,,,
+8160,1,0.00166,5.919711129991504e-06,60.5,12.957,13.133,4027.2,832,,,,
+8180,1,0.01573,5.855989804587936e-06,57.6,12.957,13.133,4038.3,824,,,,
+8200,1,0.04967,5.792268479184367e-06,58.2,12.957,13.133,4049.3,815,,,,
+8220,1,0.00287,5.728547153780799e-06,66.5,12.957,13.133,4058.9,806,,,,
+8240,1,0.01959,5.664825828377231e-06,63.9,12.957,13.133,4068.9,798,,,,
+8260,1,0.0028,5.601104502973663e-06,86.2,12.957,13.133,4076.3,788,,,,
+8280,1,0.00276,5.537383177570093e-06,92.8,12.957,13.133,4083.2,779,,,,
+8300,1,0.01501,5.473661852166525e-06,80.2,12.957,13.133,4091.2,770,,,,
+8320,1,0.04036,5.409940526762957e-06,67.8,12.957,13.133,4100.6,761,,,,
+8340,1,0.01124,5.346219201359388e-06,105.1,12.957,13.133,4106.7,751,,,,
+8360,1,0.00375,5.28249787595582e-06,62.8,12.957,13.133,4116.9,743,,,,
+8380,1,0.00737,5.218776550552252e-06,72.9,12.957,13.133,4125.7,734,,,,
+8400,1,0.00402,5.155055225148684e-06,73.3,12.957,13.133,4134.4,725,,,,
+8420,1,0.00267,5.091333899745115e-06,85.3,12.957,13.133,4141.9,715,,,,
+8440,1,0.02282,5.027612574341547e-06,74.6,12.957,13.133,4150.5,706,,,,
+8460,1,0.00256,4.9638912489379785e-06,91.0,12.957,13.133,4157.6,697,,,,
+8480,1,0.03377,4.9001699235344096e-06,63.1,12.957,13.133,4167.7,688,,,,
+8500,1,0.04083,4.836448598130841e-06,81.9,12.957,13.133,4175.5,679,,,,
+8520,1,0.01989,4.7727272727272725e-06,76.3,12.957,13.133,4183.9,670,,,,
+8540,1,0.00818,4.7090059473237045e-06,60.8,12.957,13.133,4194.4,662,,,,
+8560,1,0.00172,4.645284621920136e-06,72.4,12.957,13.133,4203.3,653,,,,
+8580,1,0.00225,4.5815632965165674e-06,61.6,12.957,13.133,4213.7,644,,,,
+8600,1,0.0141,4.517841971112999e-06,76.2,12.957,13.133,4222.0,635,,,,
+8620,1,0.00308,4.454120645709431e-06,69.3,12.957,13.133,4231.3,626,,,,
+8640,1,0.00667,4.390399320305863e-06,79.3,12.957,13.133,4239.4,617,,,,
+8660,1,0.01383,4.326677994902294e-06,64.9,12.957,13.133,4249.2,608,,,,
+8680,1,0.00309,4.262956669498725e-06,53.6,12.957,13.133,4261.2,599,,,,
+8700,1,0.02302,4.199235344095157e-06,74.6,12.957,13.133,4269.7,590,,,,
+8720,1,0.00345,4.135514018691588e-06,71.2,12.957,13.133,4278.7,582,,,,
+8740,1,0.00129,4.07179269328802e-06,79.6,12.957,13.133,4286.8,572,,,,
+8750,1,0.00231,4.039932030586237e-06,86.2,12.957,13.133,4290.5,568,0.02673,0.99373,0.99971,0.979
+8760,1,0.00653,4.008071367884452e-06,65.0,12.957,13.133,4348.9,563,,,,
+8780,1,0.01898,3.944350042480884e-06,88.7,12.957,13.133,4356.1,554,,,,
+8800,1,0.02457,3.880628717077315e-06,68.0,12.957,13.133,4365.5,545,,,,
+8820,1,0.00446,3.816907391673747e-06,100.5,12.957,13.133,4371.9,536,,,,
+8840,1,0.01478,3.7531860662701785e-06,65.6,12.957,13.133,4381.6,527,,,,
+8860,1,0.00606,3.68946474086661e-06,83.0,12.957,13.133,4389.3,518,,,,
+8880,1,0.004,3.625743415463042e-06,65.0,12.957,13.133,4399.2,509,,,,
+8900,1,0.00169,3.5620220900594734e-06,82.7,12.957,13.133,4406.9,500,,,,
+8920,1,0.00961,3.498300764655905e-06,71.9,12.957,13.133,4415.8,491,,,,
+8940,1,0.00172,3.4345794392523364e-06,63.4,12.957,13.133,4425.9,482,,,,
+8960,1,0.00232,3.3708581138487683e-06,87.0,12.957,13.133,4433.3,473,,,,
+8980,1,0.02997,3.3071367884451998e-06,63.4,12.957,13.133,4443.3,464,,,,
+9000,1,0.04303,3.2434154630416313e-06,71.3,12.957,13.133,4452.3,455,,,,
+9020,1,0.00478,3.1796941376380628e-06,77.7,12.957,13.133,4460.6,446,,,,
+9040,1,0.00081,3.1159728122344942e-06,66.8,12.957,13.133,4470.1,438,,,,
+9060,1,0.00648,3.052251486830926e-06,70.4,12.957,13.133,4479.2,429,,,,
+9080,1,0.00512,2.9885301614273577e-06,61.6,12.957,13.133,4489.6,420,,,,
+9100,1,0.02798,2.9248088360237896e-06,68.1,12.957,13.133,4499.0,411,,,,
+9120,1,0.01396,2.8610875106202206e-06,68.9,12.957,13.133,4508.3,402,,,,
+9140,1,0.00743,2.7973661852166525e-06,68.3,12.957,13.133,4517.7,393,,,,
+9160,1,0.00194,2.733644859813084e-06,60.7,12.957,13.133,4528.3,384,,,,
+9180,1,0.02428,2.669923534409516e-06,94.3,12.957,13.133,4535.0,375,,,,
+9200,1,0.00907,2.6062022090059474e-06,67.0,12.957,13.133,4544.6,366,,,,
+9220,1,0.00342,2.542480883602379e-06,58.2,12.957,13.133,4555.6,357,,,,
+9240,1,0.00271,2.4787595581988104e-06,72.2,12.957,13.133,4564.4,348,,,,
+9260,1,0.00538,2.4150382327952423e-06,65.5,12.957,13.133,4574.2,340,,,,
+9280,1,0.00069,2.351316907391674e-06,90.1,12.957,13.133,4581.3,330,,,,
+9300,1,0.00826,2.2875955819881057e-06,83.0,12.957,13.133,4589.0,321,,,,
+9320,1,0.00687,2.223874256584537e-06,67.1,12.957,13.133,4598.5,313,,,,
+9340,1,0.00792,2.1601529311809687e-06,67.4,12.957,13.133,4608.0,304,,,,
+9360,1,0.0047,2.0964316057774e-06,74.2,12.957,13.133,4616.7,295,,,,
+9380,1,0.00711,2.0327102803738317e-06,73.2,12.957,13.133,4625.4,286,,,,
+9400,1,0.00228,1.9689889549702636e-06,76.7,12.957,13.133,4633.8,277,,,,
+9420,1,0.0007,1.9052676295666949e-06,73.0,12.957,13.133,4642.5,268,,,,
+9440,1,0.00343,1.8415463041631266e-06,75.1,12.957,13.133,4651.0,259,,,,
+9460,1,0.00444,1.7778249787595583e-06,77.5,12.957,13.133,4659.3,250,,,,
+9480,1,0.00449,1.71410365335599e-06,78.0,12.957,13.133,4667.5,241,,,,
+9500,1,0.02229,1.6503823279524215e-06,67.0,12.957,13.133,4677.1,232,,,,
+9520,1,0.00556,1.5866610025488532e-06,80.7,12.957,13.133,4684.7,223,,,,
+9540,1,0.03514,1.5229396771452845e-06,75.7,12.957,13.133,4693.1,214,,,,
+9560,1,0.00626,1.4592183517417162e-06,54.1,12.957,13.133,4705.0,205,,,,
+9580,1,0.02169,1.3954970263381479e-06,79.0,12.957,13.133,4713.1,196,,,,
+9600,1,0.00201,1.3317757009345794e-06,77.9,12.957,13.133,4721.3,187,,,,
+9620,1,0.00378,1.268054375531011e-06,65.3,12.957,13.133,4731.1,178,,,,
+9640,1,0.00797,1.2043330501274426e-06,79.4,12.957,13.133,4739.1,169,,,,
+9660,1,0.00243,1.1406117247238743e-06,77.4,12.957,13.133,4747.4,160,,,,
+9680,1,0.00724,1.076890399320306e-06,83.5,12.957,13.133,4755.1,151,,,,
+9700,1,0.0003,1.0131690739167374e-06,66.0,12.957,13.133,4764.8,142,,,,
+9720,1,0.0026,9.494477485131691e-07,62.0,12.957,13.133,4775.1,133,,,,
+9740,1,0.01949,8.857264231096007e-07,63.8,12.957,13.133,4785.1,124,,,,
+9760,1,0.00144,8.220050977060323e-07,61.9,12.957,13.133,4795.5,116,,,,
+9780,1,0.02665,7.582837723024639e-07,87.4,12.957,13.133,4802.8,107,,,,
+9800,1,0.00703,6.945624468988954e-07,81.3,12.957,13.133,4810.7,98,,,,
+9820,1,0.00446,6.308411214953271e-07,64.6,12.957,13.133,4820.6,89,,,,
+9840,1,0.03959,5.671197960917587e-07,73.9,12.957,13.133,4829.3,80,,,,
+9860,1,0.05151,5.033984706881903e-07,58.5,12.957,13.133,4840.2,71,,,,
+9880,1,0.02066,4.396771452846219e-07,79.3,12.957,13.133,4848.3,62,,,,
+9900,1,0.00756,3.7595581988105356e-07,58.8,12.957,13.133,4859.2,53,,,,
+9920,1,0.00724,3.1223449447748516e-07,67.8,12.957,13.133,4868.6,44,,,,
+9940,1,0.01511,2.4851316907391675e-07,98.3,12.957,13.133,4875.1,35,,,,
+9960,1,0.01205,1.8479184367034835e-07,63.4,12.957,13.133,4885.2,26,,,,
+9980,1,0.0153,1.2107051826677997e-07,74.2,12.957,13.133,4893.8,17,,,,
+10000,1,0.00938,5.7349192863211556e-08,66.3,12.957,13.133,4903.5,8,0.02538,0.99374,0.99975,0.97748
+10018,1,0.00673,0.0,97.4,12.957,13.133,4962.7,0,0.02538,0.99374,0.99975,0.97748
diff --git a/presentation/checkpoints/pretrain/large-24L/final/config.json b/presentation/checkpoints/pretrain/large-24L/final/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..8576c3afbd43428d6205109c8dd8ec27fd85f272
--- /dev/null
+++ b/presentation/checkpoints/pretrain/large-24L/final/config.json
@@ -0,0 +1,59 @@
+{
+  "model": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 2
+  },
+  "tokenizer": "tokenizer.json",
+  "training": {
+    "run_name": "large-24L",
+    "status": "complete",
+    "source_checkpoint": "step-0009552",
+    "steps": 9552,
+    "total_steps_planned": 9552,
+    "steps_per_epoch": 796,
+    "epochs_planned": 12,
+    "epochs_completed": 12.0,
+    "tokens_seen": 1291124736,
+    "total_training_seconds": 54185.9,
+    "total_training_time": "15h 03m 05s",
+    "final_loss": 1.523698394948786,
+    "val_loss": 1.9862932266429436,
+    "val_masked_acc": 0.6291136684837473,
+    "peak_vram_reserved_gb": 17.939,
+    "peak_vram_allocated_gb": 17.494,
+    "micro_batch": 24,
+    "grad_accumulation": 11,
+    "effective_batch": 264,
+    "precision": "bf16_autocast",
+    "optimizer": "AdamW(fused) {\"lr\": 0.00015, \"betas\": [0.9, 0.98], \"eps\": 1e-06, \"weight_decay\": 0.01, \"grad_clip\": 1.0, \"mlm_probability\": 0.15, \"mask_replace_probs\": [0.8, 0.1, 0.1], \"schedule\": \"cosine_to_10pct\", \"effective_batch_sequences\": 256}",
+    "schedule": {
+      "kind": "cosine",
+      "peak_lr": 0.00015,
+      "warmup_steps": 573,
+      "total_steps": 9552,
+      "floor_ratio": 0.1
+    },
+    "mean_tokens_per_s": 23889,
+    "data": {
+      "n_windows": 210368,
+      "n_val_windows": 1999,
+      "tokenizer_sha1": "9f50c5e775089d2ca3efc82e100cd6b3396ed7e7",
+      "n_docs": 226273,
+      "limit_docs": null,
+      "window": "[CLS] + 511 stream tokens"
+    },
+    "weights_dtype": "float32",
+    "exported_at": "2026-10-09T03:31:14"
+  }
+}
diff --git a/presentation/checkpoints/pretrain/large-24L/metrics.csv b/presentation/checkpoints/pretrain/large-24L/metrics.csv
new file mode 100644
index 0000000000000000000000000000000000000000..3a4dc71d3fc66d9ef5856accc6ceff697af8c631
--- /dev/null
+++ b/presentation/checkpoints/pretrain/large-24L/metrics.csv
@@ -0,0 +1,488 @@
+step,epoch,epoch_frac,loss,loss_avg,lr,grad_norm,tokens_per_s,max_alloc_gb,max_reserved_gb,device_used_gb,elapsed_s,train_seconds_total,val_mlm_loss,val_mlm_acc
+20,1,0.0251,9.65554,10.23315,5.235602e-06,3.1232,25208,17.463,17.9,19.877,107.2,107.2,,
+40,1,0.0503,9.00961,9.23593,1.047120e-05,2.0384,25076,17.446,17.9,19.861,215.0,215.0,,
+60,1,0.0754,8.57615,8.7725,1.570681e-05,1.7805,25239,17.451,17.9,19.857,322.2,322.2,,
+80,1,0.1005,8.14762,8.37088,2.094241e-05,1.4461,25848,17.449,17.9,19.855,426.7,426.7,,
+100,1,0.1256,7.74921,7.9055,2.617801e-05,1.1524,25553,17.448,17.9,19.856,532.5,532.5,,
+120,1,0.1508,7.33637,7.47507,3.141361e-05,1.1184,24891,17.448,17.9,19.87,641.1,641.1,,
+140,1,0.1759,7.07691,7.13753,3.664921e-05,0.8894,24881,17.459,17.9,19.857,749.8,749.8,,
+160,1,0.201,6.88217,6.92639,4.188482e-05,0.8708,24949,17.472,17.9,19.862,858.2,858.2,,
+180,1,0.2261,6.84097,6.80971,4.712042e-05,1.7211,23670,17.449,17.9,19.98,972.4,972.4,,
+200,1,0.2513,6.69994,6.7201,5.235602e-05,0.7795,22891,17.453,17.9,19.963,1090.5,1090.5,,
+220,1,0.2764,6.68854,6.65768,5.759162e-05,0.8252,22869,17.448,17.9,19.969,1208.7,1208.7,,
+240,1,0.3015,6.60113,6.60204,6.282723e-05,0.7452,22885,17.442,17.9,19.967,1326.8,1326.8,,
+260,1,0.3266,6.49082,6.53755,6.806283e-05,0.8219,22936,17.45,17.9,19.931,1444.7,1444.7,,
+280,1,0.3518,6.38844,6.49443,7.329843e-05,0.8371,22884,17.457,17.92,19.949,1562.8,1562.8,,
+300,1,0.3769,6.52539,6.46607,7.853403e-05,0.7234,22863,17.466,17.92,19.949,1681.0,1681.0,,
+320,1,0.402,6.40232,6.42653,8.376963e-05,0.7215,22930,17.445,17.92,19.964,1798.9,1798.9,,
+340,1,0.4271,6.42777,6.40196,8.900524e-05,0.7596,22671,17.463,17.92,19.986,1918.2,1918.2,,
+360,1,0.4523,6.44734,6.37771,9.424084e-05,0.8369,22883,17.448,17.92,19.985,2036.3,2036.3,,
+380,1,0.4774,6.41317,6.3382,9.947644e-05,0.6934,22869,17.456,17.92,19.984,2154.5,2154.5,,
+400,1,0.5025,6.37031,6.32584,1.047120e-04,0.9135,22894,17.455,17.92,19.984,2272.6,2272.6,,
+420,1,0.5276,6.34894,6.29021,1.099476e-04,0.7507,22959,17.456,17.92,19.948,2390.4,2390.4,,
+440,1,0.5528,6.29219,6.30842,1.151832e-04,0.7328,22918,17.455,17.92,19.948,2508.3,2508.3,,
+460,1,0.5779,6.31274,6.27698,1.204188e-04,0.598,22873,17.453,17.92,19.948,2626.5,2626.5,,
+480,1,0.603,6.21508,6.24647,1.256545e-04,0.59,22883,17.447,17.92,19.953,2744.6,2744.6,,
+500,1,0.6281,6.13533,6.23042,1.308901e-04,0.6209,22911,17.453,17.92,19.96,2863.1,2863.1,6.53786,0.14698
+520,1,0.6533,6.18345,6.23754,1.361257e-04,0.5188,22962,17.461,17.92,19.954,2985.2,2985.2,,
+540,1,0.6784,6.27907,6.20101,1.413613e-04,0.6358,22917,17.456,17.92,19.946,3103.2,3103.2,,
+560,1,0.7035,6.22195,6.18063,1.465969e-04,0.6154,23015,17.459,17.92,19.95,3220.6,3220.6,,
+580,1,0.7286,6.17172,6.17752,1.499998e-04,0.5641,22941,17.455,17.92,19.949,3338.5,3338.5,,
+600,1,0.7538,6.21183,6.1728,1.499970e-04,0.7418,22910,17.448,17.92,19.946,3456.5,3456.5,,
+620,1,0.7789,6.13272,6.15495,1.499909e-04,0.5895,22888,17.466,17.92,19.943,3574.6,3574.6,,
+640,1,0.804,6.13637,6.12561,1.499815e-04,0.5007,22888,17.459,17.92,19.94,3692.7,3692.7,,
+660,1,0.8291,6.07538,6.12493,1.499687e-04,0.472,22895,17.451,17.92,19.963,3810.8,3810.8,,
+680,1,0.8543,6.07529,6.11575,1.499527e-04,0.4974,22898,17.452,17.92,19.96,3928.8,3928.8,,
+700,1,0.8794,6.11579,6.10667,1.499334e-04,0.6161,22877,17.466,17.92,19.96,4047.0,4047.0,,
+720,1,0.9045,5.98929,6.09106,1.499107e-04,0.5411,22809,17.458,17.92,19.944,4165.5,4165.5,,
+740,1,0.9296,6.09023,6.08543,1.498848e-04,0.5507,22780,17.453,17.92,19.942,4284.2,4284.2,,
+760,1,0.9548,6.11201,6.08856,1.498556e-04,0.6118,22779,17.472,17.92,19.942,4402.9,4402.9,,
+780,1,0.9799,6.04742,6.08011,1.498230e-04,0.4288,22807,17.46,17.92,19.943,4521.4,4521.4,,
+796,1,1.0,6.02973,6.09483,1.497946e-04,0.5158,22788,17.466,17.92,19.943,4616.3,4616.3,,
+800,2,1.005,5.96528,6.01872,1.497872e-04,0.4634,23085,17.425,17.92,19.938,4643.9,4643.9,,
+820,2,1.0302,6.04513,6.04329,1.497481e-04,0.4338,22867,17.448,17.92,19.94,4762.1,4762.1,,
+840,2,1.0553,5.97176,6.04791,1.497057e-04,0.4776,22775,17.462,17.92,19.937,4880.8,4880.8,,
+860,2,1.0804,6.03749,6.03486,1.496600e-04,0.4867,23091,17.459,17.92,19.916,4997.8,4997.8,,
+880,2,1.1055,6.00173,6.025,1.496110e-04,0.5696,22855,17.453,17.92,20.131,5116.1,5116.1,,
+900,2,1.1307,6.04999,6.00458,1.495587e-04,0.4447,23760,17.464,17.92,20.046,5229.9,5229.9,,
+920,2,1.1558,5.90012,6.01583,1.495031e-04,0.4157,23786,17.443,17.92,20.039,5343.6,5343.6,,
+940,2,1.1809,6.0101,6.00692,1.494443e-04,0.4874,23786,17.453,17.92,20.04,5457.2,5457.2,,
+960,2,1.206,6.04478,6.00385,1.493822e-04,0.4532,23771,17.454,17.92,20.039,5570.9,5570.9,,
+980,2,1.2312,6.04821,6.00704,1.493168e-04,0.4345,23912,17.448,17.92,20.005,5684.0,5684.0,,
+1000,2,1.2563,6.02353,6.01251,1.492481e-04,0.5543,23897,17.455,17.92,19.963,5797.6,5797.6,6.38459,0.15324
+1020,2,1.2814,5.98558,5.98435,1.491762e-04,0.4935,24010,17.451,17.92,19.965,5916.1,5916.1,,
+1040,2,1.3065,5.89927,5.97594,1.491009e-04,0.4365,23940,17.455,17.92,19.968,6029.0,6029.0,,
+1060,2,1.3317,5.97554,5.98769,1.490225e-04,0.4352,23784,17.455,17.92,19.969,6142.7,6142.7,,
+1080,2,1.3568,6.00287,5.98597,1.489408e-04,0.4299,23844,17.46,17.92,19.976,6256.1,6256.1,,
+1100,2,1.3819,5.96485,5.97473,1.488558e-04,0.4503,23921,17.462,17.92,19.972,6369.1,6369.1,,
+1120,2,1.407,5.94334,5.96578,1.487676e-04,0.454,23888,17.445,17.92,19.96,6482.2,6482.2,,
+1140,2,1.4322,5.97784,5.95465,1.486761e-04,0.4512,23878,17.45,17.92,19.956,6595.5,6595.5,,
+1160,2,1.4573,6.00912,5.95672,1.485814e-04,0.4994,23936,17.464,17.939,19.975,6708.4,6708.4,,
+1180,2,1.4824,5.95698,5.96522,1.484834e-04,0.4333,23923,17.472,17.939,19.975,6821.4,6821.4,,
+1200,2,1.5075,5.96124,5.92618,1.483823e-04,0.7809,23936,17.457,17.939,19.977,6934.3,6934.3,,
+1220,2,1.5327,5.79389,5.83968,1.482779e-04,0.8617,23940,17.457,17.939,19.975,7047.3,7047.3,,
+1240,2,1.5578,5.70641,5.78812,1.481702e-04,0.6042,23957,17.475,17.939,19.977,7160.1,7160.1,,
+1260,2,1.5829,5.73939,5.73052,1.480594e-04,0.7582,24036,17.463,17.939,19.989,7272.6,7272.6,,
+1280,2,1.608,5.68666,5.6825,1.479453e-04,0.7121,24076,17.46,17.939,19.976,7384.9,7384.9,,
+1300,2,1.6332,5.63178,5.64273,1.478281e-04,0.7001,24167,17.458,17.939,19.98,7496.7,7496.7,,
+1320,2,1.6583,5.61608,5.6198,1.477076e-04,1.5807,24183,17.47,17.939,19.978,7608.5,7608.5,,
+1340,2,1.6834,5.61262,5.59723,1.475840e-04,1.2622,24192,17.458,17.939,19.978,7720.3,7720.3,,
+1360,2,1.7085,5.55659,5.55062,1.474571e-04,1.4437,24183,17.464,17.939,19.978,7832.1,7832.1,,
+1380,2,1.7337,5.53974,5.53853,1.473271e-04,1.2723,24191,17.448,17.939,19.978,7943.8,7943.8,,
+1400,2,1.7588,5.41222,5.52107,1.471939e-04,1.0183,24240,17.451,17.939,19.978,8055.3,8055.3,,
+1420,2,1.7839,5.40574,5.49235,1.470576e-04,1.0306,24263,17.459,17.939,19.978,8166.7,8166.7,,
+1440,2,1.809,5.45829,5.46803,1.469181e-04,1.2285,24294,17.474,17.939,19.98,8278.0,8278.0,,
+1460,2,1.8342,5.42616,5.43396,1.467754e-04,2.0296,24272,17.449,17.939,19.978,8389.4,8389.4,,
+1480,2,1.8593,5.32151,5.42887,1.466296e-04,1.2172,24293,17.47,17.939,19.976,8500.7,8500.7,,
+1500,2,1.8844,5.47039,5.42365,1.464806e-04,0.7762,24299,17.455,17.939,19.977,8612.4,8612.4,5.80502,0.18696
+1520,2,1.9095,5.38949,5.36718,1.463285e-04,1.2464,24301,17.445,17.939,19.98,8726.9,8726.9,,
+1540,2,1.9347,5.2836,5.33209,1.461733e-04,1.9915,24287,17.46,17.939,19.978,8838.2,8838.2,,
+1560,2,1.9598,5.30269,5.32253,1.460150e-04,1.8347,24177,17.46,17.939,20.036,8950.0,8950.0,,
+1580,2,1.9849,5.26982,5.31419,1.458535e-04,1.922,24201,17.45,17.939,20.051,9061.7,9061.7,,
+1592,2,2.0,5.25049,5.27551,1.457552e-04,1.4867,24248,17.457,17.939,20.047,9128.6,9128.6,,
+1600,3,2.0101,5.37491,5.25221,1.456890e-04,1.862,24227,17.464,17.939,20.109,9176.8,9176.8,,
+1620,3,2.0352,5.12937,5.24556,1.455213e-04,1.481,24245,17.47,17.939,20.072,9288.3,9288.3,,
+1640,3,2.0603,5.13487,5.24114,1.453506e-04,1.236,24248,17.455,17.939,20.065,9399.8,9399.8,,
+1660,3,2.0854,5.08512,5.209,1.451768e-04,1.6116,24351,17.472,17.939,20.059,9510.8,9510.8,,
+1680,3,2.1106,5.21874,5.18663,1.449999e-04,1.8846,24328,17.446,17.939,20.044,9621.9,9621.9,,
+1700,3,2.1357,5.18338,5.15426,1.448200e-04,0.8621,24351,17.46,17.939,20.041,9733.0,9733.0,,
+1720,3,2.1608,5.12875,5.15041,1.446370e-04,1.361,24324,17.463,17.939,20.025,9844.1,9844.1,,
+1740,3,2.1859,5.04259,5.13464,1.444510e-04,1.6264,24337,17.457,17.939,20.024,9955.2,9955.2,,
+1760,3,2.2111,4.97737,5.09574,1.442619e-04,2.0622,24365,17.457,17.939,20.024,10066.1,10066.1,,
+1780,3,2.2362,4.93518,5.0781,1.440698e-04,1.4842,24359,17.451,17.939,20.023,10177.1,10177.1,,
+1800,3,2.2613,4.97431,5.05867,1.438747e-04,1.8452,24337,17.46,17.939,20.024,10288.2,10288.2,,
+1820,3,2.2864,4.9922,5.0229,1.436766e-04,1.5323,24339,17.442,17.939,20.026,10399.2,10399.2,,
+1840,3,2.3116,4.93831,5.00883,1.434755e-04,1.7726,24338,17.456,17.939,20.023,10510.3,10510.3,,
+1860,3,2.3367,4.9443,4.96859,1.432714e-04,1.8008,24349,17.456,17.939,20.025,10621.4,10621.4,,
+1880,3,2.3618,4.93528,4.96363,1.430643e-04,1.6559,24348,17.471,17.939,20.023,10732.4,10732.4,,
+1900,3,2.3869,4.90354,4.93146,1.428543e-04,1.6981,24352,17.45,17.939,20.023,10843.4,10843.4,,
+1920,3,2.4121,4.87819,4.90077,1.426413e-04,1.7199,24366,17.494,17.939,20.026,10954.3,10954.3,,
+1940,3,2.4372,4.85458,4.88391,1.424254e-04,1.7752,24355,17.454,17.939,20.023,11065.3,11065.3,,
+1960,3,2.4623,4.91053,4.85662,1.422065e-04,1.7343,24339,17.448,17.939,20.023,11176.4,11176.4,,
+1980,3,2.4874,4.79094,4.81827,1.419847e-04,1.1959,24349,17.45,17.939,20.023,11287.4,11287.4,,
+2000,3,2.5126,4.80719,4.82767,1.417600e-04,1.1155,24339,17.466,17.939,20.025,11398.9,11398.9,5.24622,0.24712
+2020,3,2.5377,4.76848,4.80792,1.415324e-04,1.5519,24298,17.464,17.939,20.024,11514.0,11514.0,,
+2040,3,2.5628,4.75923,4.74817,1.413019e-04,2.7,24263,17.454,17.939,20.024,11625.4,11625.4,,
+2060,3,2.5879,4.81154,4.72938,1.410686e-04,1.7255,24257,17.456,17.939,20.025,11736.8,11736.8,,
+2080,3,2.6131,4.76066,4.67963,1.408323e-04,1.4753,24275,17.457,17.939,20.024,11848.2,11848.2,,
+2100,3,2.6382,4.56537,4.65587,1.405932e-04,1.2697,24282,17.47,17.939,20.027,11959.5,11959.5,,
+2120,3,2.6633,4.63027,4.61135,1.403513e-04,1.2496,24267,17.474,17.939,20.025,12070.9,12070.9,,
+2140,3,2.6884,4.47353,4.53668,1.401065e-04,1.3424,24280,17.448,17.939,20.023,12182.2,12182.2,,
+2160,3,2.7136,4.5525,4.51125,1.398589e-04,1.0971,24271,17.452,17.939,20.024,12293.6,12293.6,,
+2180,3,2.7387,4.39373,4.39575,1.396085e-04,1.1104,24278,17.447,17.939,20.025,12405.0,12405.0,,
+2200,3,2.7638,4.30037,4.29726,1.393553e-04,1.3021,24291,17.456,17.939,20.025,12516.3,12516.3,,
+2220,3,2.7889,4.24514,4.24236,1.390993e-04,1.4855,24261,17.454,17.939,20.022,12627.7,12627.7,,
+2240,3,2.8141,4.13817,4.17799,1.388406e-04,1.277,24146,17.452,17.939,19.998,12739.7,12739.7,,
+2260,3,2.8392,4.08045,4.10739,1.385791e-04,1.3312,24126,17.45,17.939,19.998,12851.7,12851.7,,
+2280,3,2.8643,4.01112,4.06048,1.383148e-04,0.958,24121,17.458,17.939,19.998,12963.8,12963.8,,
+2300,3,2.8894,4.0017,4.04082,1.380478e-04,1.498,24113,17.453,17.939,19.998,13075.9,13075.9,,
+2320,3,2.9146,3.87894,3.9572,1.377781e-04,1.1703,24137,17.451,17.939,19.998,13187.9,13187.9,,
+2340,3,2.9397,3.84486,3.90693,1.375057e-04,0.9503,24071,17.457,17.939,19.997,13300.2,13300.2,,
+2360,3,2.9648,3.88792,3.85636,1.372305e-04,1.0208,24073,17.45,17.939,19.999,13412.5,13412.5,,
+2380,3,2.9899,3.79513,3.79729,1.369528e-04,0.8979,24054,17.447,17.939,20.001,13524.9,13524.9,,
+2388,3,3.0,3.73439,3.73687,1.368409e-04,0.8805,24064,17.443,17.939,19.999,13569.8,13569.8,,
+2400,4,3.0151,3.83922,3.69856,1.366723e-04,0.8462,24128,17.449,17.939,19.999,13640.6,13640.6,,
+2420,4,3.0402,3.62159,3.62287,1.363892e-04,0.9918,24069,17.454,17.939,19.999,13752.9,13752.9,,
+2440,4,3.0653,3.49245,3.5729,1.361034e-04,0.9559,24066,17.452,17.939,19.999,13865.2,13865.2,,
+2460,4,3.0905,3.60922,3.51618,1.358150e-04,1.021,24039,17.452,17.939,19.998,13977.7,13977.7,,
+2480,4,3.1156,3.39781,3.46561,1.355241e-04,0.7809,24048,17.453,17.939,19.999,14090.1,14090.1,,
+2500,4,3.1407,3.34572,3.42977,1.352305e-04,0.7629,24060,17.445,17.939,19.999,14202.9,14202.9,3.90752,0.39209
+2520,4,3.1658,3.52306,3.41779,1.349343e-04,0.877,24042,17.456,17.939,20.0,14319.4,14319.4,,
+2540,4,3.191,3.25502,3.34914,1.346356e-04,0.7136,23420,17.462,17.939,20.243,14434.9,14434.9,,
+2560,4,3.2161,3.21455,3.31935,1.343343e-04,0.7678,23144,17.456,17.939,20.263,14551.7,14551.7,,
+2580,4,3.2412,3.35182,3.27418,1.340305e-04,0.7685,23589,17.449,17.939,20.201,14666.3,14666.3,,
+2600,4,3.2663,3.15693,3.24513,1.337241e-04,0.7807,23731,17.455,17.939,20.188,14780.2,14780.2,,
+2620,4,3.2915,3.17677,3.20748,1.334153e-04,0.7325,23784,17.464,17.939,20.156,14893.8,14893.8,,
+2640,4,3.3166,3.12428,3.16251,1.331039e-04,0.7412,23963,17.445,17.939,20.144,15006.7,15006.7,,
+2660,4,3.3417,3.20963,3.144,1.327901e-04,0.7045,23954,17.438,17.939,20.143,15119.5,15119.5,,
+2680,4,3.3668,3.08086,3.09518,1.324738e-04,0.7971,23978,17.451,17.939,20.156,15232.3,15232.3,,
+2700,4,3.392,3.07921,3.07384,1.321550e-04,0.7227,23956,17.458,17.939,20.155,15345.1,15345.1,,
+2720,4,3.4171,2.95923,3.0476,1.318339e-04,0.7203,23974,17.453,17.939,20.155,15457.9,15457.9,,
+2740,4,3.4422,3.00622,3.0373,1.315103e-04,0.7386,23967,17.457,17.939,20.157,15570.7,15570.7,,
+2760,4,3.4673,2.96494,3.00408,1.311843e-04,0.7284,23973,17.462,17.939,20.155,15683.4,15683.4,,
+2780,4,3.4925,2.97857,2.97997,1.308559e-04,0.7439,23982,17.447,17.939,20.154,15796.2,15796.2,,
+2800,4,3.5176,2.94309,2.95262,1.305252e-04,0.7225,23994,17.451,17.939,20.155,15908.8,15908.8,,
+2820,4,3.5427,2.92279,2.92127,1.301921e-04,0.7362,24013,17.45,17.939,20.156,16021.4,16021.4,,
+2840,4,3.5678,2.84159,2.89189,1.298567e-04,0.7327,23983,17.459,17.939,20.155,16134.1,16134.1,,
+2860,4,3.593,2.77116,2.86046,1.295190e-04,0.7105,23990,17.451,17.939,20.155,16246.8,16246.8,,
+2880,4,3.6181,2.90426,2.82809,1.291789e-04,0.7091,23994,17.457,17.939,20.155,16359.5,16359.5,,
+2900,4,3.6432,2.84675,2.82805,1.288366e-04,0.7203,23985,17.469,17.939,20.157,16472.2,16472.2,,
+2920,4,3.6683,2.77801,2.82633,1.284920e-04,0.7263,23988,17.456,17.939,20.155,16584.9,16584.9,,
+2940,4,3.6935,2.8213,2.80071,1.281451e-04,0.7183,23966,17.452,17.939,20.157,16697.7,16697.7,,
+2960,4,3.7186,2.79464,2.76543,1.277960e-04,0.7176,23992,17.465,17.939,20.156,16810.4,16810.4,,
+2980,4,3.7437,2.66469,2.74876,1.274447e-04,0.7026,24005,17.459,17.939,20.155,16923.0,16923.0,,
+3000,4,3.7688,2.5925,2.72507,1.270912e-04,0.6937,23979,17.447,17.939,20.155,17036.2,17036.2,3.15992,0.4783
+3020,4,3.794,2.62328,2.72059,1.267355e-04,0.7213,24005,17.456,17.939,20.155,17152.4,17152.4,,
+3040,4,3.8191,2.67507,2.67976,1.263777e-04,0.6879,23988,17.448,17.939,20.156,17265.1,17265.1,,
+3060,4,3.8442,2.77888,2.67849,1.260177e-04,0.7363,23966,17.464,17.939,20.155,17377.9,17377.9,,
+3080,4,3.8693,2.62588,2.66832,1.256556e-04,0.72,23970,17.452,17.939,20.155,17490.7,17490.7,,
+3100,4,3.8945,2.70497,2.6488,1.252913e-04,0.7088,23975,17.449,17.939,20.156,17603.5,17603.5,,
+3120,4,3.9196,2.57143,2.61979,1.249250e-04,0.692,24008,17.451,17.939,20.154,17716.1,17716.1,,
+3140,4,3.9447,2.60178,2.60699,1.245565e-04,0.7211,23950,17.457,17.939,20.153,17828.9,17828.9,,
+3160,4,3.9698,2.60105,2.59293,1.241861e-04,0.6963,23960,17.466,17.939,20.154,17941.8,17941.8,,
+3180,4,3.995,2.56745,2.5947,1.238135e-04,0.7306,23978,17.453,17.939,20.154,18054.5,18054.5,,
+3184,4,4.0,2.50733,2.55136,1.237388e-04,0.696,23952,17.437,17.939,20.156,18077.1,18077.1,,
+3200,5,4.0201,2.52249,2.52659,1.234390e-04,0.6841,24000,17.452,17.939,20.155,18170.5,18170.5,,
+3220,5,4.0452,2.52334,2.55331,1.230624e-04,0.6994,23970,17.444,17.939,20.155,18283.3,18283.3,,
+3240,5,4.0704,2.48262,2.52429,1.226839e-04,0.6911,23941,17.458,17.939,20.157,18396.2,18396.2,,
+3260,5,4.0955,2.58836,2.55783,1.223034e-04,0.6966,23931,17.471,17.939,20.143,18509.2,18509.2,,
+3280,5,4.1206,2.48034,2.51958,1.219210e-04,0.696,23941,17.448,17.939,20.142,18622.1,18622.1,,
+3300,5,4.1457,2.5611,2.50252,1.215366e-04,0.7207,23927,17.466,17.939,20.141,18735.1,18735.1,,
+3320,5,4.1709,2.51432,2.49113,1.211503e-04,0.7248,23925,17.459,17.939,20.142,18848.1,18848.1,,
+3340,5,4.196,2.47976,2.45571,1.207621e-04,0.6845,23906,17.453,17.939,20.145,18961.2,18961.2,,
+3360,5,4.2211,2.53939,2.4414,1.203720e-04,0.6815,23920,17.445,17.939,20.143,19074.2,19074.2,,
+3380,5,4.2462,2.4455,2.43914,1.199801e-04,0.7168,23889,17.446,17.939,20.144,19187.4,19187.4,,
+3400,5,4.2714,2.43887,2.43887,1.195864e-04,0.6814,23901,17.456,17.939,20.145,19300.5,19300.5,,
+3420,5,4.2965,2.43342,2.43368,1.191908e-04,0.6781,23887,17.471,17.939,20.143,19413.6,19413.6,,
+3440,5,4.3216,2.39886,2.40732,1.187934e-04,0.6634,23882,17.449,17.939,20.145,19526.8,19526.8,,
+3460,5,4.3467,2.42741,2.40387,1.183943e-04,0.682,23842,17.466,17.939,20.195,19640.2,19640.2,,
+3480,5,4.3719,2.31299,2.39693,1.179934e-04,0.6908,23878,17.469,17.939,20.177,19753.4,19753.4,,
+3500,5,4.397,2.43548,2.38571,1.175908e-04,0.6785,23868,17.458,17.939,20.178,19867.1,19867.1,2.78485,0.52998
+3520,5,4.4221,2.39857,2.36852,1.171864e-04,0.676,23994,17.452,17.939,20.178,19983.5,19983.5,,
+3540,5,4.4472,2.44255,2.34269,1.167804e-04,0.6754,23897,17.446,17.939,20.161,20096.6,20096.6,,
+3560,5,4.4724,2.24196,2.37863,1.163726e-04,0.6704,23856,17.46,17.939,20.159,20210.0,20210.0,,
+3580,5,4.4975,2.27098,2.33233,1.159633e-04,0.6614,23866,17.465,17.939,20.159,20323.2,20323.2,,
+3600,5,4.5226,2.32216,2.33279,1.155522e-04,0.6536,23846,17.447,17.939,20.161,20436.6,20436.6,,
+3620,5,4.5477,2.28032,2.34131,1.151396e-04,0.6422,23853,17.449,17.939,20.159,20549.9,20549.9,,
+3640,5,4.5729,2.31836,2.30676,1.147253e-04,0.6656,23890,17.459,17.939,20.16,20663.1,20663.1,,
+3660,5,4.598,2.33368,2.29587,1.143095e-04,0.6764,23877,17.46,17.939,20.161,20776.3,20776.3,,
+3680,5,4.6231,2.28177,2.30056,1.138921e-04,0.6481,23853,17.475,17.939,20.16,20889.6,20889.6,,
+3700,5,4.6482,2.27864,2.30483,1.134732e-04,0.6532,23854,17.468,17.939,20.158,21003.0,21003.0,,
+3720,5,4.6734,2.27947,2.26538,1.130528e-04,0.6581,23852,17.45,17.939,20.158,21116.3,21116.3,,
+3740,5,4.6985,2.21925,2.27018,1.126309e-04,0.6429,23854,17.45,17.939,20.161,21229.6,21229.6,,
+3760,5,4.7236,2.26316,2.25689,1.122075e-04,0.6359,23871,17.463,17.939,20.159,21342.9,21342.9,,
+3780,5,4.7487,2.25515,2.25942,1.117826e-04,0.6798,23827,17.461,17.939,20.16,21456.3,21456.3,,
+3800,5,4.7739,2.22701,2.24882,1.113563e-04,0.6539,23860,17.456,17.939,20.161,21569.6,21569.6,,
+3820,5,4.799,2.28195,2.23956,1.109286e-04,0.6617,23832,17.459,17.939,20.162,21683.1,21683.1,,
+3840,5,4.8241,2.24882,2.24016,1.104995e-04,0.6477,23844,17.451,17.939,20.161,21796.5,21796.5,,
+3860,5,4.8492,2.18558,2.22474,1.100690e-04,0.6423,23858,17.45,17.939,20.16,21909.8,21909.8,,
+3880,5,4.8744,2.27608,2.23895,1.096372e-04,0.6675,23854,17.464,17.939,20.159,22023.1,22023.1,,
+3900,5,4.8995,2.27267,2.21442,1.092041e-04,0.6516,23851,17.451,17.939,20.159,22136.4,22136.4,,
+3920,5,4.9246,2.25982,2.23283,1.087696e-04,0.6585,23855,17.474,17.939,20.159,22249.8,22249.8,,
+3940,5,4.9497,2.18325,2.21415,1.083339e-04,0.6562,23833,17.459,17.939,20.161,22363.2,22363.2,,
+3960,5,4.9749,2.19043,2.16752,1.078969e-04,0.64,23835,17.453,17.939,20.159,22476.6,22476.6,,
+3980,5,5.0,2.14873,2.16911,1.074586e-04,0.6547,23825,17.462,17.939,20.159,22590.1,22590.1,,
+4000,6,5.0251,2.19222,2.15876,1.070191e-04,0.6443,23900,17.453,17.939,20.161,22707.3,22707.3,2.58341,0.54835
+4020,6,5.0503,2.28927,2.14716,1.065785e-04,0.6748,23876,17.459,17.939,20.16,22824.3,22824.3,,
+4040,6,5.0754,2.14925,2.17546,1.061366e-04,0.6241,23853,17.447,17.939,20.157,22937.6,22937.6,,
+4060,6,5.1005,2.17078,2.16374,1.056936e-04,0.6372,23842,17.463,17.939,20.16,23051.0,23051.0,,
+4080,6,5.1256,2.14503,2.14593,1.052494e-04,0.6558,23832,17.449,17.939,20.159,23164.4,23164.4,,
+4100,6,5.1508,2.23374,2.12448,1.048042e-04,0.6462,23833,17.462,17.939,20.161,23277.9,23277.9,,
+4120,6,5.1759,2.16064,2.11934,1.043578e-04,0.6513,23839,17.48,17.939,20.159,23391.3,23391.3,,
+4140,6,5.201,2.24861,2.12185,1.039104e-04,0.6564,23874,17.439,17.939,20.159,23504.5,23504.5,,
+4160,6,5.2261,2.12293,2.12734,1.034619e-04,0.6231,23837,17.45,17.939,20.161,23617.9,23617.9,,
+4180,6,5.2513,2.08642,2.12061,1.030124e-04,0.6335,23845,17.45,17.939,20.159,23731.3,23731.3,,
+4200,6,5.2764,2.2009,2.14629,1.025619e-04,0.6369,23863,17.451,17.939,20.159,23844.6,23844.6,,
+4220,6,5.3015,2.16057,2.08631,1.021105e-04,0.6472,23852,17.463,17.939,20.159,23957.9,23957.9,,
+4240,6,5.3266,2.20065,2.10776,1.016580e-04,0.6566,23850,17.447,17.939,20.158,24071.3,24071.3,,
+4260,6,5.3518,2.03737,2.09191,1.012046e-04,0.6467,23839,17.449,17.939,20.158,24184.7,24184.7,,
+4280,6,5.3769,2.09526,2.09896,1.007503e-04,0.6314,23846,17.442,17.939,20.161,24298.0,24298.0,,
+4300,6,5.402,2.07704,2.09023,1.002951e-04,0.6328,23606,17.455,17.939,20.211,24412.5,24412.5,,
+4320,6,5.4271,2.08029,2.08686,9.983908e-05,0.6256,23627,17.465,17.939,20.198,24527.0,24527.0,,
+4340,6,5.4523,2.1213,2.06667,9.938217e-05,0.6645,23881,17.453,17.939,20.196,24640.2,24640.2,,
+4360,6,5.4774,2.14086,2.0751,9.892443e-05,0.6448,23859,17.459,17.939,20.199,24753.5,24753.5,,
+4380,6,5.5025,2.05963,2.06401,9.846589e-05,0.641,23885,17.467,17.939,20.161,24866.6,24866.6,,
+4400,6,5.5276,1.98218,2.06003,9.800656e-05,0.6232,23853,17.458,17.939,20.161,24980.0,24980.0,,
+4420,6,5.5528,2.09337,2.06746,9.754648e-05,0.6283,23868,17.46,17.939,20.163,25093.2,25093.2,,
+4440,6,5.5779,2.12878,2.03468,9.708566e-05,0.6306,23844,17.452,17.939,20.181,25206.6,25206.6,,
+4460,6,5.603,1.95475,2.04623,9.662412e-05,0.6144,23846,17.454,17.939,20.165,25320.0,25320.0,,
+4480,6,5.6281,1.9855,2.03448,9.616189e-05,0.6117,23833,17.449,17.939,20.162,25433.4,25433.4,,
+4500,6,5.6533,2.07512,2.02161,9.569900e-05,0.6322,23860,17.454,17.939,20.163,25547.2,25547.2,2.44039,0.56854
+4520,6,5.6784,2.005,2.04053,9.523545e-05,0.6321,23911,17.45,17.939,20.164,25664.0,25664.0,,
+4540,6,5.7035,2.02198,2.01152,9.477129e-05,0.6262,23857,17.461,17.939,20.145,25777.3,25777.3,,
+4560,6,5.7286,2.02487,2.02087,9.430652e-05,0.6345,23884,17.451,17.939,20.145,25890.5,25890.5,,
+4580,6,5.7538,1.93431,2.00891,9.384118e-05,0.6155,23989,17.455,17.939,20.145,26003.2,26003.2,,
+4600,6,5.7789,1.96699,2.01033,9.337528e-05,0.6239,24011,17.45,17.939,20.145,26115.8,26115.8,,
+4620,6,5.804,2.00033,1.9946,9.290885e-05,0.6264,24021,17.443,17.939,20.145,26228.3,26228.3,,
+4640,6,5.8291,2.00339,1.9938,9.244190e-05,0.6157,24012,17.464,17.939,20.146,26340.9,26340.9,,
+4660,6,5.8543,2.05247,1.97949,9.197447e-05,0.6346,23915,17.455,17.939,20.163,26453.9,26453.9,,
+4680,6,5.8794,1.90607,1.98331,9.150658e-05,0.6213,23986,17.451,17.939,20.143,26566.6,26566.6,,
+4700,6,5.9045,2.05909,2.00184,9.103825e-05,0.6429,24028,17.454,17.939,20.145,26679.1,26679.1,,
+4720,6,5.9296,1.91958,2.00151,9.056949e-05,0.6213,24008,17.463,17.939,20.147,26791.7,26791.7,,
+4740,6,5.9548,1.95518,1.97532,9.010035e-05,0.6292,24030,17.474,17.939,20.145,26904.2,26904.2,,
+4760,6,5.9799,1.93146,1.9877,8.963083e-05,0.6326,24013,17.444,17.939,20.145,27016.8,27016.8,,
+4776,6,6.0,2.00279,1.98212,8.925496e-05,0.6295,24030,17.44,17.939,20.145,27106.8,27106.8,,
+4780,7,6.005,1.95653,1.94948,8.916096e-05,0.6153,24369,17.44,17.939,20.145,27133.0,27133.0,,
+4800,7,6.0302,1.92787,1.93602,8.869076e-05,0.6263,24182,17.436,17.939,20.143,27244.8,27244.8,,
+4820,7,6.0553,2.03857,1.97808,8.822027e-05,0.631,24065,17.471,17.939,20.147,27357.1,27357.1,,
+4840,7,6.0804,1.97294,1.95994,8.774949e-05,0.628,24067,17.454,17.939,20.145,27469.5,27469.5,,
+4860,7,6.1055,1.9637,1.97453,8.727845e-05,0.6469,24098,17.458,17.939,20.145,27581.6,27581.6,,
+4880,7,6.1307,1.87169,1.93362,8.680718e-05,0.6258,24135,17.454,17.939,20.146,27693.7,27693.7,,
+4900,7,6.1558,1.97796,1.94919,8.633570e-05,0.6288,24174,17.451,17.939,20.153,27805.5,27805.5,,
+4920,7,6.1809,1.96137,1.92641,8.586404e-05,0.6428,24163,17.446,17.939,20.153,27917.4,27917.4,,
+4940,7,6.206,1.95144,1.94613,8.539220e-05,0.6283,24199,17.465,17.939,20.151,28029.1,28029.1,,
+4960,7,6.2312,1.99066,1.90504,8.492023e-05,0.6283,24218,17.461,17.939,20.153,28140.7,28140.7,,
+4980,7,6.2563,1.95508,1.94733,8.444814e-05,0.6186,24242,17.441,17.939,20.152,28252.2,28252.2,,
+5000,7,6.2814,1.92299,1.93634,8.397595e-05,0.6435,24216,17.445,17.939,20.153,28364.3,28364.3,2.35576,0.58066
+5020,7,6.3065,1.92456,1.91454,8.350369e-05,0.6254,24349,17.455,17.939,20.152,28479.1,28479.1,,
+5040,7,6.3317,1.88008,1.91799,8.303138e-05,0.6249,24236,17.461,17.939,20.152,28590.6,28590.6,,
+5060,7,6.3568,1.8487,1.90847,8.255904e-05,0.6168,24221,17.479,17.939,20.155,28702.2,28702.2,,
+5080,7,6.3819,1.86189,1.90574,8.208670e-05,0.6215,24226,17.463,17.939,20.153,28813.8,28813.8,,
+5100,7,6.407,1.88789,1.91614,8.161439e-05,0.6206,24222,17.453,17.939,20.151,28925.4,28925.4,,
+5120,7,6.4322,1.87132,1.8858,8.114211e-05,0.6196,24225,17.448,17.939,20.153,29037.0,29037.0,,
+5140,7,6.4573,1.88296,1.91214,8.066990e-05,0.6453,24232,17.456,17.939,20.153,29148.6,29148.6,,
+5160,7,6.4824,1.84918,1.90195,8.019778e-05,0.6228,24245,17.46,17.939,20.153,29260.1,29260.1,,
+5180,7,6.5075,1.96947,1.90848,7.972578e-05,0.6305,24211,17.447,17.939,20.189,29371.7,29371.7,,
+5200,7,6.5327,1.91409,1.8994,7.925391e-05,0.6288,24248,17.457,17.939,20.191,29483.2,29483.2,,
+5220,7,6.5578,1.93347,1.89024,7.878220e-05,0.6291,24198,17.469,17.939,20.189,29595.0,29595.0,,
+5240,7,6.5829,1.96567,1.89046,7.831067e-05,0.6436,24219,17.464,17.939,20.187,29706.6,29706.6,,
+5260,7,6.608,1.86378,1.89115,7.783934e-05,0.617,23059,17.472,17.939,20.2,29823.8,29823.8,,
+5280,7,6.6332,1.93988,1.86004,7.736825e-05,0.6373,23892,17.448,17.939,20.122,29937.0,29937.0,,
+5300,7,6.6583,1.88952,1.892,7.689740e-05,0.6399,23143,17.444,17.939,20.1,30053.8,30053.8,,
+5320,7,6.6834,1.9116,1.87554,7.642683e-05,0.6297,23380,17.455,17.939,20.146,30169.4,30169.4,,
+5340,7,6.7085,1.75656,1.86794,7.595656e-05,0.616,23366,17.453,17.939,20.175,30285.1,30285.1,,
+5360,7,6.7337,1.88941,1.86231,7.548661e-05,0.6307,23585,17.445,17.939,20.092,30399.7,30399.7,,
+5380,7,6.7588,1.93091,1.86731,7.501700e-05,0.638,23310,17.475,17.939,20.047,30515.7,30515.7,,
+5400,7,6.7839,1.90185,1.86655,7.454776e-05,0.633,23070,17.452,17.939,20.028,30632.9,30632.9,,
+5420,7,6.809,1.87184,1.8695,7.407890e-05,0.6267,23139,17.446,17.939,20.016,30749.7,30749.7,,
+5440,7,6.8342,1.82629,1.86102,7.361046e-05,0.6423,22482,17.443,17.939,20.139,30869.9,30869.9,,
+5460,7,6.8593,1.8483,1.84741,7.314246e-05,0.6299,22512,17.437,17.939,20.085,30990.0,30990.0,,
+5480,7,6.8844,1.88856,1.83832,7.267491e-05,0.6404,23079,17.448,17.939,19.994,31107.2,31107.2,,
+5500,7,6.9095,1.81683,1.84004,7.220784e-05,0.6196,23240,17.47,17.939,19.974,31223.9,31223.9,2.26972,0.59116
+5520,7,6.9347,1.80016,1.84828,7.174128e-05,0.6145,23321,17.451,17.939,19.975,31345.9,31345.9,,
+5540,7,6.9598,1.88758,1.84607,7.127525e-05,0.6405,23240,17.448,17.939,19.954,31462.3,31462.3,,
+5560,7,6.9849,1.95252,1.83343,7.080976e-05,0.6449,23245,17.456,17.939,20.031,31578.6,31578.6,,
+5572,7,7.0,1.7931,1.81582,7.053074e-05,0.6152,23229,17.462,17.939,20.018,31648.4,31648.4,,
+5580,8,7.0101,1.77449,1.7756,7.034485e-05,0.6155,23337,17.439,17.939,20.017,31698.8,31698.8,,
+5600,8,7.0352,1.9037,1.82183,6.988053e-05,0.6344,22494,17.445,17.939,20.079,31818.9,31818.9,,
+5620,8,7.0603,1.86663,1.80757,6.941683e-05,0.638,22690,17.46,17.939,20.08,31938.1,31938.1,,
+5640,8,7.0854,1.77126,1.822,6.895377e-05,0.6376,22642,17.464,17.939,20.069,32057.5,32057.5,,
+5660,8,7.1106,1.84426,1.80915,6.849137e-05,0.6292,22677,17.445,17.939,20.066,32176.7,32176.7,,
+5680,8,7.1357,1.76639,1.81848,6.802966e-05,0.6129,22682,17.465,17.939,20.068,32295.9,32295.9,,
+5700,8,7.1608,1.76659,1.80443,6.756866e-05,0.6134,22656,17.446,17.939,20.167,32415.2,32415.2,,
+5720,8,7.1859,1.87748,1.80846,6.710839e-05,0.6308,22155,17.458,17.939,20.074,32537.2,32537.2,,
+5740,8,7.2111,1.71967,1.80842,6.664887e-05,0.6353,22994,17.462,17.939,20.283,32654.8,32654.8,,
+5760,8,7.2362,1.84665,1.80172,6.619013e-05,0.6409,22536,17.456,17.939,20.175,32774.7,32774.7,,
+5780,8,7.2613,1.82295,1.81435,6.573219e-05,0.6205,23202,17.45,17.939,20.07,32891.3,32891.3,,
+5800,8,7.2864,1.76973,1.7937,6.527507e-05,0.6321,22950,17.467,17.939,20.08,33009.0,33009.0,,
+5820,8,7.3116,1.7642,1.77742,6.481879e-05,0.6411,22969,17.456,17.939,20.012,33126.7,33126.7,,
+5840,8,7.3367,1.7736,1.79388,6.436338e-05,0.6335,23403,17.458,17.939,20.017,33242.3,33242.3,,
+5860,8,7.3618,1.87632,1.7968,6.390886e-05,0.6235,23990,17.447,17.939,20.018,33354.9,33354.9,,
+5880,8,7.3869,1.68472,1.77706,6.345524e-05,0.6146,24000,17.465,17.939,20.03,33467.6,33467.6,,
+5900,8,7.4121,1.8202,1.81452,6.300256e-05,0.6292,23687,17.444,17.939,20.237,33581.7,33581.7,,
+5920,8,7.4372,1.72633,1.79557,6.255084e-05,0.6287,23747,17.463,17.939,20.263,33695.6,33695.6,,
+5940,8,7.4623,1.76493,1.7676,6.210009e-05,0.6176,23029,17.457,17.939,20.276,33812.9,33812.9,,
+5960,8,7.4874,1.85276,1.79257,6.165034e-05,0.6321,22232,17.451,17.939,20.307,33934.5,33934.5,,
+5980,8,7.5126,1.80302,1.78176,6.120161e-05,0.6333,22219,17.446,17.939,20.278,34056.2,34056.2,,
+6000,8,7.5377,1.77444,1.78666,6.075392e-05,0.6311,22539,17.447,17.939,20.285,34176.6,34176.6,2.1979,0.59964
+6020,8,7.5628,1.83465,1.77488,6.030730e-05,0.6424,22777,17.457,17.939,20.36,34299.0,34299.0,,
+6040,8,7.5879,1.73046,1.76923,5.986177e-05,0.6252,23207,17.447,17.939,20.233,34415.4,34415.4,,
+6060,8,7.6131,1.7713,1.7722,5.941734e-05,0.6325,23304,17.442,17.939,20.259,34531.5,34531.5,,
+6080,8,7.6382,1.7427,1.76502,5.897404e-05,0.6261,23594,17.458,17.939,20.281,34646.0,34646.0,,
+6100,8,7.6633,1.77987,1.77677,5.853190e-05,0.6435,22520,17.459,17.939,20.308,34766.1,34766.1,,
+6120,8,7.6884,1.7539,1.77477,5.809093e-05,0.6342,22677,17.462,17.939,20.461,34885.3,34885.3,,
+6140,8,7.7136,1.82006,1.74577,5.765115e-05,0.6251,23162,17.458,17.939,20.257,35002.0,35002.0,,
+6160,8,7.7387,1.84485,1.74753,5.721260e-05,0.6422,22678,17.459,17.939,20.537,35121.2,35121.2,,
+6180,8,7.7638,1.78851,1.76432,5.677527e-05,0.646,22170,17.462,17.939,20.289,35243.1,35243.1,,
+6200,8,7.7889,1.68273,1.73674,5.633921e-05,0.6264,22856,17.449,17.939,20.332,35361.4,35361.4,,
+6220,8,7.8141,1.7672,1.75681,5.590444e-05,0.6208,23394,17.452,17.939,20.525,35477.0,35477.0,,
+6240,8,7.8392,1.81878,1.75672,5.547096e-05,0.6541,23683,17.45,17.939,20.476,35591.1,35591.1,,
+6260,8,7.8643,1.68474,1.73901,5.503880e-05,0.6222,24094,17.454,17.939,20.26,35703.3,35703.3,,
+6280,8,7.8894,1.74759,1.75089,5.460800e-05,0.6346,24255,17.459,17.939,20.26,35814.8,35814.8,,
+6300,8,7.9146,1.71824,1.73387,5.417855e-05,0.6279,24300,17.449,17.939,20.222,35926.0,35926.0,,
+6320,8,7.9397,1.85975,1.74499,5.375050e-05,0.6383,24340,17.449,17.939,20.22,36037.1,36037.1,,
+6340,8,7.9648,1.82879,1.75504,5.332385e-05,0.6407,24349,17.45,17.939,20.219,36148.1,36148.1,,
+6360,8,7.9899,1.72864,1.73886,5.289863e-05,0.6344,24353,17.453,17.939,20.222,36259.1,36259.1,,
+6368,8,8.0,1.77658,1.73384,5.272894e-05,0.6414,24358,17.449,17.939,20.219,36303.5,36303.5,,
+6380,9,8.0151,1.7564,1.7269,5.247486e-05,0.6269,24366,17.441,17.939,20.219,36373.7,36373.7,,
+6400,9,8.0402,1.74679,1.6996,5.205256e-05,0.631,24379,17.455,17.939,20.22,36484.6,36484.6,,
+6420,9,8.0653,1.73622,1.72735,5.163175e-05,0.6255,24330,17.465,17.939,20.22,36595.7,36595.7,,
+6440,9,8.0905,1.6725,1.7208,5.121245e-05,0.619,24264,17.44,17.939,20.219,36707.1,36707.1,,
+6460,9,8.1156,1.74703,1.71636,5.079468e-05,0.6341,24227,17.451,17.939,20.219,36818.7,36818.7,,
+6480,9,8.1407,1.66339,1.71223,5.037847e-05,0.6324,24226,17.475,17.939,20.221,36930.3,36930.3,,
+6500,9,8.1658,1.79373,1.71101,4.996383e-05,0.6465,24220,17.461,17.939,20.222,37042.4,37042.4,2.14395,0.61094
+6520,9,8.191,1.72958,1.71882,4.955078e-05,0.6518,24254,17.447,17.939,20.222,37157.3,37157.3,,
+6540,9,8.2161,1.70496,1.70816,4.913935e-05,0.6306,24190,17.45,17.939,20.222,37269.0,37269.0,,
+6560,9,8.2412,1.70678,1.72454,4.872955e-05,0.641,24183,17.441,17.939,20.223,37380.8,37380.8,,
+6580,9,8.2663,1.66562,1.72073,4.832140e-05,0.624,24182,17.445,17.939,20.222,37492.6,37492.6,,
+6600,9,8.2915,1.73815,1.7099,4.791493e-05,0.6345,24192,17.448,17.939,20.222,37604.3,37604.3,,
+6620,9,8.3166,1.68391,1.70952,4.751015e-05,0.6376,24176,17.465,17.939,20.224,37716.2,37716.2,,
+6640,9,8.3417,1.66753,1.68883,4.710708e-05,0.6404,24174,17.456,17.939,20.224,37828.0,37828.0,,
+6660,9,8.3668,1.70916,1.69916,4.670575e-05,0.6447,24161,17.456,17.939,20.222,37939.9,37939.9,,
+6680,9,8.392,1.71971,1.68607,4.630617e-05,0.6364,24154,17.453,17.939,20.222,38051.8,38051.8,,
+6700,9,8.4171,1.66924,1.69469,4.590836e-05,0.644,24157,17.443,17.939,20.222,38163.7,38163.7,,
+6720,9,8.4422,1.68874,1.67793,4.551235e-05,0.6405,24136,17.468,17.939,20.222,38275.7,38275.7,,
+6740,9,8.4673,1.7402,1.70891,4.511814e-05,0.6298,24136,17.449,17.939,20.224,38387.7,38387.7,,
+6760,9,8.4925,1.7364,1.71264,4.472577e-05,0.6542,24141,17.442,17.939,20.207,38499.7,38499.7,,
+6780,9,8.5176,1.71812,1.70878,4.433524e-05,0.6375,24142,17.454,17.939,20.207,38611.7,38611.7,,
+6800,9,8.5427,1.70483,1.68534,4.394659e-05,0.6475,24139,17.449,17.939,20.208,38723.7,38723.7,,
+6820,9,8.5678,1.71152,1.68656,4.355982e-05,0.6381,24141,17.453,17.939,20.207,38835.6,38835.6,,
+6840,9,8.593,1.67343,1.68666,4.317496e-05,0.637,24141,17.466,17.939,20.207,38947.6,38947.6,,
+6860,9,8.6181,1.73496,1.69975,4.279202e-05,0.6469,24113,17.464,17.939,20.208,39059.7,39059.7,,
+6880,9,8.6432,1.80288,1.71175,4.241103e-05,0.6374,24141,17.455,17.939,20.207,39171.7,39171.7,,
+6900,9,8.6683,1.67858,1.70167,4.203200e-05,0.6402,24154,17.455,17.939,20.209,39283.6,39283.6,,
+6920,9,8.6935,1.68381,1.68707,4.165495e-05,0.6318,24193,17.456,17.939,20.207,39395.4,39395.4,,
+6940,9,8.7186,1.72237,1.67464,4.127991e-05,0.6283,24220,17.459,17.939,20.21,39507.0,39507.0,,
+6960,9,8.7437,1.55671,1.67776,4.090688e-05,0.6115,24282,17.444,17.939,20.208,39618.3,39618.3,,
+6980,9,8.7688,1.70912,1.70213,4.053589e-05,0.645,24304,17.461,17.939,20.209,39729.6,39729.6,,
+7000,9,8.794,1.62386,1.67213,4.016695e-05,0.6258,24336,17.464,17.939,20.208,39841.1,39841.1,2.11306,0.61155
+7020,9,8.8191,1.78215,1.6744,3.980009e-05,0.6469,24372,17.45,17.939,20.219,39955.8,39955.8,,
+7040,9,8.8442,1.63802,1.66142,3.943531e-05,0.6213,24333,17.458,17.939,20.218,40066.9,40066.9,,
+7060,9,8.8693,1.70556,1.66583,3.907265e-05,0.6402,24336,17.446,17.939,20.218,40178.0,40178.0,,
+7080,9,8.8945,1.63843,1.67262,3.871211e-05,0.6338,24327,17.457,17.939,20.22,40289.2,40289.2,,
+7100,9,8.9196,1.74019,1.66731,3.835372e-05,0.675,24269,17.454,17.939,20.218,40400.6,40400.6,,
+7120,9,8.9447,1.60957,1.64556,3.799749e-05,0.6357,24216,17.456,17.939,20.218,40512.2,40512.2,,
+7140,9,8.9698,1.67903,1.66779,3.764343e-05,0.639,24209,17.464,17.939,20.218,40623.9,40623.9,,
+7160,9,8.995,1.69586,1.67626,3.729158e-05,0.646,24198,17.461,17.939,20.22,40735.6,40735.6,,
+7164,9,9.0,1.58374,1.65247,3.722147e-05,0.6314,24208,17.436,17.939,20.22,40757.9,40757.9,,
+7180,10,9.0201,1.66971,1.67264,3.694194e-05,0.6418,24271,17.455,17.939,20.22,40850.9,40850.9,,
+7200,10,9.0452,1.68371,1.65745,3.659452e-05,0.64,24196,17.451,17.939,20.22,40962.6,40962.6,,
+7220,10,9.0704,1.66862,1.65207,3.624936e-05,0.6286,24193,17.46,17.939,20.22,41074.4,41074.4,,
+7240,10,9.0955,1.64013,1.656,3.590646e-05,0.6422,24195,17.46,17.939,20.222,41186.1,41186.1,,
+7260,10,9.1206,1.67634,1.64639,3.556584e-05,0.6505,24178,17.461,17.939,20.22,41297.9,41297.9,,
+7280,10,9.1457,1.72853,1.66066,3.522752e-05,0.6513,24190,17.45,17.939,20.22,41409.7,41409.7,,
+7300,10,9.1709,1.70544,1.66083,3.489152e-05,0.6412,24184,17.457,17.939,20.24,41521.4,41521.4,,
+7320,10,9.196,1.65399,1.63852,3.455785e-05,0.6373,24188,17.44,17.939,20.238,41633.2,41633.2,,
+7340,10,9.2211,1.75565,1.65338,3.422652e-05,0.657,24182,17.455,17.939,20.239,41745.0,41745.0,,
+7360,10,9.2462,1.62372,1.64228,3.389756e-05,0.632,24191,17.463,17.939,20.238,41856.7,41856.7,,
+7380,10,9.2714,1.66384,1.65985,3.357098e-05,0.638,24182,17.465,17.939,20.24,41968.5,41968.5,,
+7400,10,9.2965,1.66145,1.6467,3.324679e-05,0.6456,24166,17.452,17.939,20.238,42080.4,42080.4,,
+7420,10,9.3216,1.63032,1.62209,3.292502e-05,0.64,24194,17.46,17.939,20.236,42192.1,42192.1,,
+7440,10,9.3467,1.66156,1.65127,3.260567e-05,0.6335,24195,17.455,17.939,20.236,42303.9,42303.9,,
+7460,10,9.3719,1.6636,1.63861,3.228876e-05,0.6532,24151,17.456,17.939,20.236,42415.8,42415.8,,
+7480,10,9.397,1.78286,1.65077,3.197432e-05,0.6558,24116,17.447,17.939,20.238,42527.9,42527.9,,
+7500,10,9.4221,1.75807,1.63424,3.166235e-05,0.6711,24130,17.449,17.939,20.237,42640.4,42640.4,2.07307,0.61841
+7520,10,9.4472,1.59945,1.64389,3.135287e-05,0.6273,24178,17.448,17.939,20.224,42756.0,42756.0,,
+7540,10,9.4724,1.71637,1.649,3.104589e-05,0.6501,24189,17.462,17.939,20.223,42867.8,42867.8,,
+7560,10,9.4975,1.68983,1.62896,3.074143e-05,0.6541,24133,17.448,17.939,20.223,42979.8,42979.8,,
+7580,10,9.5226,1.62055,1.62931,3.043951e-05,0.6346,24115,17.456,17.939,20.225,43091.9,43091.9,,
+7600,10,9.5477,1.6946,1.63176,3.014014e-05,0.6499,24107,17.455,17.939,20.223,43204.1,43204.1,,
+7620,10,9.5729,1.67784,1.61237,2.984333e-05,0.6562,24126,17.461,17.939,20.223,43316.1,43316.1,,
+7640,10,9.598,1.61661,1.62795,2.954909e-05,0.6477,24126,17.452,17.939,20.222,43428.2,43428.2,,
+7660,10,9.6231,1.64756,1.63847,2.925746e-05,0.662,24125,17.452,17.939,20.223,43540.2,43540.2,,
+7680,10,9.6482,1.61584,1.62211,2.896842e-05,0.6354,24126,17.46,17.939,20.224,43652.3,43652.3,,
+7700,10,9.6734,1.63772,1.62881,2.868201e-05,0.641,24122,17.461,17.939,20.223,43764.3,43764.3,,
+7720,10,9.6985,1.61556,1.63076,2.839824e-05,0.655,24117,17.447,17.939,20.225,43876.4,43876.4,,
+7740,10,9.7236,1.64742,1.6092,2.811711e-05,0.6464,24113,17.46,17.939,20.222,43988.5,43988.5,,
+7760,10,9.7487,1.5433,1.61729,2.783865e-05,0.6488,24121,17.453,17.939,20.224,44100.6,44100.6,,
+7780,10,9.7739,1.61388,1.63942,2.756286e-05,0.6503,24132,17.45,17.939,20.223,44212.6,44212.6,,
+7800,10,9.799,1.6921,1.61221,2.728977e-05,0.6607,24122,17.46,17.939,20.227,44324.7,44324.7,,
+7820,10,9.8241,1.60186,1.60806,2.701938e-05,0.6326,24137,17.481,17.939,20.225,44436.7,44436.7,,
+7840,10,9.8492,1.63476,1.62944,2.675170e-05,0.6548,24121,17.445,17.939,20.227,44548.8,44548.8,,
+7860,10,9.8744,1.5897,1.62895,2.648675e-05,0.6307,24125,17.448,17.939,20.225,44660.8,44660.8,,
+7880,10,9.8995,1.63954,1.63001,2.622455e-05,0.6487,24121,17.455,17.939,20.225,44772.9,44772.9,,
+7900,10,9.9246,1.60526,1.61756,2.596510e-05,0.6463,24068,17.443,17.939,20.227,44885.2,44885.2,,
+7920,10,9.9497,1.68403,1.60666,2.570842e-05,0.6575,24087,17.453,17.939,20.227,44997.5,44997.5,,
+7940,10,9.9749,1.64991,1.60952,2.545452e-05,0.6512,24101,17.464,17.939,20.223,45109.6,45109.6,,
+7960,10,10.0,1.64678,1.60766,2.520342e-05,0.6555,24097,17.449,17.939,20.223,45221.8,45221.8,,
+7980,11,10.0251,1.63676,1.59274,2.495512e-05,0.6403,24181,17.462,17.939,20.223,45341.7,45341.7,,
+8000,11,10.0503,1.61543,1.60596,2.470964e-05,0.6572,24128,17.456,17.939,20.223,45454.2,45454.2,2.041,0.62225
+8020,11,10.0754,1.50024,1.57838,2.446699e-05,0.6506,24182,17.452,17.939,20.223,45574.0,45574.0,,
+8040,11,10.1005,1.58824,1.59886,2.422718e-05,0.6477,24128,17.457,17.939,20.223,45686.0,45686.0,,
+8060,11,10.1256,1.61041,1.6068,2.399022e-05,0.6442,24123,17.459,17.939,20.225,45798.1,45798.1,,
+8080,11,10.1508,1.61475,1.60058,2.375613e-05,0.6456,24130,17.458,17.939,20.223,45910.1,45910.1,,
+8100,11,10.1759,1.622,1.60855,2.352491e-05,0.6563,24120,17.47,17.939,20.223,46022.2,46022.2,,
+8120,11,10.201,1.54648,1.60135,2.329658e-05,0.6421,24116,17.473,17.939,20.225,46134.3,46134.3,,
+8140,11,10.2261,1.56605,1.62373,2.307116e-05,0.6328,24106,17.454,17.939,20.223,46246.4,46246.4,,
+8160,11,10.2513,1.64682,1.61611,2.284864e-05,0.6543,24103,17.458,17.939,20.223,46358.6,46358.6,,
+8180,11,10.2764,1.56365,1.6,2.262904e-05,0.6468,24117,17.453,17.939,20.226,46470.7,46470.7,,
+8200,11,10.3015,1.54284,1.57427,2.241238e-05,0.6554,24124,17.454,17.939,20.223,46582.7,46582.7,,
+8220,11,10.3266,1.61902,1.59868,2.219865e-05,0.6557,24115,17.457,17.939,20.225,46694.8,46694.8,,
+8240,11,10.3518,1.62747,1.61804,2.198788e-05,0.6663,24088,17.476,17.939,20.223,46807.1,46807.1,,
+8260,11,10.3769,1.60535,1.59738,2.178007e-05,0.6563,24113,17.446,17.939,20.224,46919.2,46919.2,,
+8280,11,10.402,1.62032,1.60679,2.157524e-05,0.6616,24123,17.449,17.939,20.223,47031.2,47031.2,,
+8300,11,10.4271,1.59546,1.60778,2.137339e-05,0.6519,24138,17.451,17.939,20.223,47143.2,47143.2,,
+8320,11,10.4523,1.68314,1.59721,2.117453e-05,0.6664,24136,17.448,17.939,20.223,47255.2,47255.2,,
+8340,11,10.4774,1.64062,1.59642,2.097868e-05,0.6675,24117,17.445,17.939,20.223,47367.3,47367.3,,
+8360,11,10.5025,1.53298,1.61639,2.078583e-05,0.6505,24134,17.457,17.939,20.223,47479.4,47479.4,,
+8380,11,10.5276,1.59435,1.57717,2.059601e-05,0.658,24120,17.462,17.939,20.225,47591.4,47591.4,,
+8400,11,10.5528,1.61873,1.60246,2.040922e-05,0.6559,24130,17.458,17.939,20.223,47703.5,47703.5,,
+8420,11,10.5779,1.62925,1.58609,2.022547e-05,0.66,24140,17.447,17.939,20.223,47815.5,47815.5,,
+8440,11,10.603,1.55886,1.58155,2.004477e-05,0.6627,24128,17.45,17.939,20.225,47927.5,47927.5,,
+8460,11,10.6281,1.51416,1.59196,1.986713e-05,0.6364,24130,17.458,17.939,20.225,48039.5,48039.5,,
+8480,11,10.6533,1.60334,1.58745,1.969256e-05,0.6542,24116,17.46,17.939,20.227,48151.6,48151.6,,
+8500,11,10.6784,1.57519,1.58576,1.952106e-05,0.654,24126,17.446,17.939,20.225,48264.1,48264.1,2.02503,0.62407
+8520,11,10.7035,1.56051,1.59532,1.935265e-05,0.6622,24126,17.455,17.939,20.225,48381.5,48381.5,,
+8540,11,10.7286,1.55205,1.59135,1.918732e-05,0.6481,24121,17.461,17.939,20.228,48493.5,48493.5,,
+8560,11,10.7538,1.56855,1.60034,1.902510e-05,0.6602,24113,17.458,17.939,20.225,48605.6,48605.6,,
+8580,11,10.7789,1.64929,1.56575,1.886599e-05,0.6661,24128,17.458,17.939,20.227,48717.7,48717.7,,
+8600,11,10.804,1.61924,1.58258,1.870999e-05,0.6549,24124,17.447,17.939,20.225,48829.7,48829.7,,
+8620,11,10.8291,1.51481,1.5843,1.855711e-05,0.6475,24119,17.463,17.939,20.226,48941.8,48941.8,,
+8640,11,10.8543,1.61418,1.58576,1.840737e-05,0.6622,24087,17.454,17.939,20.225,49054.1,49054.1,,
+8660,11,10.8794,1.64476,1.59419,1.826076e-05,0.6679,24096,17.483,17.939,20.225,49166.3,49166.3,,
+8680,11,10.9045,1.56349,1.57365,1.811730e-05,0.6561,24114,17.472,17.939,20.227,49278.4,49278.4,,
+8700,11,10.9296,1.66806,1.58655,1.797700e-05,0.657,24125,17.467,17.939,20.225,49390.4,49390.4,,
+8720,11,10.9548,1.57616,1.57976,1.783985e-05,0.6525,24126,17.458,17.939,20.225,49502.5,49502.5,,
+8740,11,10.9799,1.59715,1.57469,1.770587e-05,0.6591,24130,17.449,17.939,20.225,49614.5,49614.5,,
+8756,11,11.0,1.56513,1.55969,1.760097e-05,0.6555,24134,17.448,17.939,20.227,49704.1,49704.1,,
+8760,12,11.005,1.60617,1.58444,1.757506e-05,0.6535,24096,17.446,17.939,20.225,49731.4,49731.4,,
+8780,12,11.0302,1.5856,1.58147,1.744743e-05,0.6645,24130,17.458,17.939,20.223,49843.4,49843.4,,
+8800,12,11.0553,1.57117,1.57494,1.732298e-05,0.658,24120,17.451,17.939,20.223,49955.5,49955.5,,
+8820,12,11.0804,1.56485,1.57035,1.720173e-05,0.6541,24112,17.453,17.939,20.225,50067.6,50067.6,,
+8840,12,11.1055,1.53842,1.56621,1.708368e-05,0.6577,24118,17.452,17.939,20.223,50179.7,50179.7,,
+8860,12,11.1307,1.6753,1.57344,1.696883e-05,0.6667,24126,17.455,17.939,20.225,50291.7,50291.7,,
+8880,12,11.1558,1.63558,1.5814,1.685718e-05,0.665,24129,17.453,17.939,20.223,50403.8,50403.8,,
+8900,12,11.1809,1.52244,1.57623,1.674875e-05,0.6499,24113,17.452,17.939,20.223,50515.9,50515.9,,
+8920,12,11.206,1.5166,1.57263,1.664354e-05,0.6422,24129,17.453,17.939,20.223,50627.9,50627.9,,
+8940,12,11.2312,1.5943,1.57241,1.654156e-05,0.6561,24095,17.449,17.939,20.224,50740.1,50740.1,,
+8960,12,11.2563,1.55064,1.57602,1.644281e-05,0.6605,24095,17.459,17.939,20.223,50852.3,50852.3,,
+8980,12,11.2814,1.54096,1.58073,1.634729e-05,0.6511,24125,17.445,17.939,20.224,50964.4,50964.4,,
+9000,12,11.3065,1.55601,1.5841,1.625500e-05,0.6585,24109,17.446,17.939,20.226,51076.9,51076.9,2.00504,0.62932
+9020,12,11.3317,1.62304,1.55179,1.616597e-05,0.664,24097,17.441,17.939,20.224,51193.9,51193.9,,
+9040,12,11.3568,1.53469,1.57957,1.608018e-05,0.6558,23995,17.458,17.939,20.202,51306.5,51306.5,,
+9060,12,11.3819,1.55987,1.57074,1.599764e-05,0.6607,24077,17.445,17.939,20.203,51418.8,51418.8,,
+9080,12,11.407,1.53679,1.54607,1.591836e-05,0.6467,24088,17.457,17.939,20.259,51531.0,51531.0,,
+9100,12,11.4322,1.55332,1.5748,1.584234e-05,0.6462,24109,17.466,17.939,20.261,51643.2,51643.2,,
+9120,12,11.4573,1.56842,1.56844,1.576959e-05,0.6564,24104,17.461,17.939,20.259,51755.3,51755.3,,
+9140,12,11.4824,1.57802,1.57483,1.570010e-05,0.6495,24097,17.457,17.939,20.222,51867.5,51867.5,,
+9160,12,11.5075,1.5889,1.57764,1.563388e-05,0.675,24112,17.461,17.939,20.22,51979.6,51979.6,,
+9180,12,11.5327,1.45973,1.5647,1.557094e-05,0.6455,24093,17.452,17.939,20.22,52091.8,52091.8,,
+9200,12,11.5578,1.58689,1.56439,1.551127e-05,0.6678,24101,17.458,17.939,20.22,52204.0,52204.0,,
+9220,12,11.5829,1.63766,1.57029,1.545489e-05,0.6622,24092,17.479,17.939,20.22,52316.2,52316.2,,
+9240,12,11.608,1.52319,1.56865,1.540179e-05,0.6578,24089,17.446,17.939,20.22,52428.4,52428.4,,
+9260,12,11.6332,1.53766,1.56083,1.535197e-05,0.6561,24092,17.458,17.939,20.22,52540.6,52540.6,,
+9280,12,11.6583,1.51591,1.57931,1.530544e-05,0.6551,24087,17.464,17.939,20.22,52652.9,52652.9,,
+9300,12,11.6834,1.56069,1.54609,1.526220e-05,0.6589,24086,17.458,17.939,20.222,52765.1,52765.1,,
+9320,12,11.7085,1.5526,1.55921,1.522226e-05,0.6594,24085,17.448,17.939,20.22,52877.3,52877.3,,
+9340,12,11.7337,1.56059,1.57796,1.518561e-05,0.6649,24086,17.459,17.939,20.223,52989.6,52989.6,,
+9360,12,11.7588,1.56444,1.56113,1.515225e-05,0.6711,24085,17.443,17.939,20.22,53101.8,53101.8,,
+9380,12,11.7839,1.59458,1.55478,1.512219e-05,0.6688,24084,17.452,17.939,20.22,53214.1,53214.1,,
+9400,12,11.809,1.55706,1.56042,1.509543e-05,0.6538,24010,17.448,17.939,20.223,53326.7,53326.7,,
+9420,12,11.8342,1.55047,1.55778,1.507198e-05,0.6544,24067,17.453,17.939,20.22,53439.0,53439.0,,
+9440,12,11.8593,1.56366,1.56404,1.505182e-05,0.6763,24059,17.469,17.939,20.222,53551.4,53551.4,,
+9460,12,11.8844,1.62798,1.54293,1.503497e-05,0.6659,24064,17.447,17.939,20.22,53663.7,53663.7,,
+9480,12,11.9095,1.50276,1.56752,1.502142e-05,0.6604,24069,17.453,17.939,20.222,53776.0,53776.0,,
+9500,12,11.9347,1.52827,1.54985,1.501117e-05,0.6588,24087,17.454,17.939,20.22,53888.7,53888.7,1.98837,0.62972
+9520,12,11.9598,1.62311,1.57236,1.500423e-05,0.6737,24076,17.467,17.939,20.22,54005.4,54005.4,,
+9540,12,11.9849,1.56714,1.57561,1.500059e-05,0.6666,24024,17.474,17.939,20.267,54117.9,54117.9,,
+9552,12,12.0,1.5237,1.57567,1.500000e-05,0.6621,24032,17.454,17.939,20.266,54185.9,54185.9,1.98629,0.62911
diff --git a/presentation/checkpoints/pretrain/large-24L/run_summary.json b/presentation/checkpoints/pretrain/large-24L/run_summary.json
new file mode 100644
index 0000000000000000000000000000000000000000..750e87f5334e4544c81bc11bc7431c91b7b5262a
--- /dev/null
+++ b/presentation/checkpoints/pretrain/large-24L/run_summary.json
@@ -0,0 +1,91 @@
+{
+  "step": 9552,
+  "epoch": 12,
+  "position": 0,
+  "tokens_seen": 1291124736,
+  "train_seconds_total": 54185.850963340024,
+  "peak_reserved_gb": 17.939453125,
+  "peak_allocated_gb": 17.494221687316895,
+  "last_loss": 1.523698394948786,
+  "last_val_loss": 1.9862932266429436,
+  "last_val_acc": 0.6291136684837473,
+  "invocations": 1,
+  "status": "complete",
+  "exit_code": 0,
+  "run_end_step": 9552,
+  "this_invocation": {
+    "steps": 9552,
+    "tokens": 1291124736,
+    "loop_seconds": 54190.0,
+    "mean_tokens_per_s": 23889,
+    "setup_seconds": 1.7
+  },
+  "mean_tokens_per_s": 23889,
+  "config": {
+    "model": {
+      "vocab_size": 32768,
+      "max_seq_len": 512,
+      "d_model": 1024,
+      "n_layers": 24,
+      "n_heads": 16,
+      "d_ff": 4096,
+      "dropout": 0.1,
+      "attention_dropout": 0.1,
+      "layer_norm_eps": 1e-05,
+      "pad_token_id": 0,
+      "tie_mlm_head": true,
+      "num_labels": 2
+    },
+    "grad_checkpointing": false,
+    "pretrain": {
+      "lr": 0.00015,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "weight_decay": 0.01,
+      "grad_clip": 1.0,
+      "mlm_probability": 0.15,
+      "mask_replace_probs": [
+        0.8,
+        0.1,
+        0.1
+      ],
+      "schedule": "cosine_to_10pct",
+      "effective_batch_sequences": 256
+    },
+    "seed": 42,
+    "micro_batch": 24,
+    "grad_accumulation": 11,
+    "effective_batch": 264,
+    "schedule": {
+      "kind": "cosine",
+      "peak_lr": 0.00015,
+      "warmup_steps": 573,
+      "total_steps": 9552,
+      "floor_ratio": 0.1
+    },
+    "steps_per_epoch": 796,
+    "epochs": 12,
+    "total_steps": 9552,
+    "data": {
+      "n_windows": 210368,
+      "n_val_windows": 1999,
+      "tokenizer_sha1": "9f50c5e775089d2ca3efc82e100cd6b3396ed7e7",
+      "n_docs": 226273,
+      "limit_docs": null,
+      "window": "[CLS] + 511 stream tokens"
+    }
+  },
+  "params": {
+    "total": 337476608,
+    "trainable": 337476608,
+    "embeddings": 34080768,
+    "encoder_layers": 302309376,
+    "final_norm": 2048,
+    "head": 1084416
+  },
+  "finished": "2026-10-09T03:31:14",
+  "log": "/home/nathank/BITS/dev/Phishing Model/logs/pretrain-20261008-122802.log"
+}
diff --git a/presentation/config.yaml b/presentation/config.yaml
new file mode 100644
index 0000000000000000000000000000000000000000..36f987fe5642ae34fe213be530e26dfec70f4f41
--- /dev/null
+++ b/presentation/config.yaml
@@ -0,0 +1,221 @@
+# BITS Phishing Detector — all dials in one place.
+# Verified pins and the reasoning behind every number: docs/research/*.md (2026-10-07).
+
+seed: 42
+
+paths:
+  data_raw: data/raw
+  data_processed: data/processed
+  tokenizer: data/processed/tokenizer.json
+  checkpoints: checkpoints
+  reports: reports
+  logs: logs
+  app_dir: app
+
+# ---------------------------------------------------------------- model (Step 3)
+model:
+  # "Large" BERT-style pre-LN encoder. 24L/1024/16/4096/32768/512 = 337,476,608 params with the
+  # tied MLM head, 337,443,842 as the deployed classifier (+2.3% over the ~330M target).
+  # Switch n_layers to 23 (324,880,384) only if ~330M must be a hard ceiling.
+  n_layers: 24
+  d_model: 1024
+  n_heads: 16            # head_dim 64 -> Flash-SDPA eligible on sm_86
+  d_ff: 4096
+  vocab_size: 32768      # 2^15, tensor-core aligned
+  max_seq_len: 512
+  dropout: 0.1
+  attention_dropout: 0.1
+  norm: pre_ln
+  activation: gelu
+  tie_mlm_head: true
+  grad_checkpointing: false   # B=16 fits without it; checkpointing costs ~1.33x compute
+
+# ---------------------------------------------------------------- tokenizer
+tokenizer:
+  type: byte_level_bpe
+  vocab_size: 32768
+  min_frequency: 2
+  special_tokens: ["[PAD]", "[UNK]", "[CLS]", "[SEP]", "[MASK]"]
+  # Trained ONLY on the pretraining corpus + fine-tune train split. Never on val/test.
+  train_on: [pretrain, finetune_train]
+
+# ---------------------------------------------------------------- data (Step 2 / data_prep)
+data:
+  labelled_primary: puyang2025/seven-phishing-email-datasets   # 203,017 rows; text/subject/label/dataset_name
+  labelled_excluded:
+    - it4lia/PhishingEmailCuratedDatasets_Cleaned   # no body text (subjects + 53 numeric features only)
+    - zefang-liu/phishing-email-dataset             # 91.9% exact subset of the primary set, lossy text
+  pretrain_corpus: corbt/enron-emails               # 517,401 rows -> ~244k after exact dedup
+  holdout_test_source: TREC-06       # never seen in tokenizer, pretraining, or fine-tuning
+  second_choice_holdout: CEAS-08     # only after stripping mailing-list footers
+  # Positive class = label 1 of the primary set ("phishing or spam / unwanted"). All label-1 rows
+  # come from spam-track corpora with no spam-vs-phishing distinction; mapping spam->0 would leave
+  # ~0 positives. PENDING Nathan's confirmation (see RUN_LOG.md entry 0).
+  spam_policy: positive_class
+  min_alnum_chars: 20              # labelled rows: min alphanumerics in subject+body
+  pretrain_min_alnum_chars: 50     # corbt bodies shorter than this (after quote stripping) are dropped
+  quote_strip_min_alnum: 200       # cut a quoted/forwarded tail only if this much text survives above it
+  dedup:
+    normalize: "NFKC; lowercase; keep [a-z0-9] only (common.normalize_text)"
+    keys: [full_sha1, prefix300_sha1, subject_body_prefix300_sha1, subject_body_full_sha1]
+    key_prefix_chars: 300
+    key_prefix_min_len: 80
+    near_dup_shingle_chars: 50
+    near_dup_stride: 25
+    near_dup_min_shared: 3
+    near_dup_window_chars: 1500
+    near_dup_shingle_cap: 2000     # shingles shared by more rows than this are boilerplate; ignored and counted
+  finetune_val_fraction: 0.10        # grouped by near-dup cluster, stratified by source x label
+  model_input: "subject + \"\\n\\n\" + body"   # never sender/receiver (TREC headers are synthetic)
+  synthetic:
+    enabled: false                   # <= 2,000 rows, source=synthetic, training-only; results reported with/without
+    max_rows: 2000
+  # Three-way labels (RUN_LOG entry 6; data_prep.py build3 / check3). label3: 0 legitimate / 1 phishing / 2 spam /
+  # -1 unwanted-but-unsplit; label_set: legit / phish / spam / unwanted. The binary `label` column is unchanged.
+  threeway:
+    # Provenance of a binary-positive row -> label3. TREC-05/06/07 and CEAS-08 never separated spam from phishing
+    # (-1: trains on P(phishing)+P(spam)); SpamAssassin / Enron-Spam / Ling label-1 rows are spam folders (2);
+    # Nazario and Nigerian_Fraud are phishing corpora by construction (1). Every label-0 row is 0.
+    positive_label3_by_source:
+      Assassin: 2
+      Enron: 2
+      Ling: 2
+      TREC-05: -1
+      TREC-06: -1
+      TREC-07: -1
+      CEAS-08: -1
+      Nazario: 1
+      Nigerian_Fraud: 1
+    champa:                          # Champa et al. raw corpus mirror (CC-BY-4.0, byte-identical to Zenodo 8339691)
+      repo: kudzaiprichard/aura-phishing-email-corpus
+      files: [Nazario.csv, Nigerian_Fraud.csv]
+      local_dir: data/raw/aura
+      label3: 1
+    bigshp:                          # locuoco 300k set: research verdict DO_NOT_USE (docs/research/bigshp_research.md)
+      enabled: false                 # set true only on a USE / USE_PARTIALLY verdict; the filter chain below then applies
+      verdict: DO_NOT_USE
+      repo: locuoco/the-biggest-spam-ham-phish-email-dataset-300000
+      file: df.csv
+      local_dir: data/raw/bigshp
+      drop_phish_class: true         # its phish label is source-derived (87% spam-folder / spam-track mail) and stopword-stripped
+      min_chars: 160                 # SMS-like fragments
+      min_english_fraction: 0.7      # label-2 rows: share of word tokens in our legitimate-mail vocabulary
+      near_dup_token_key_len: 30     # within-file near-dup key: first 30 word tokens
+    new_source_test_fraction: 0.15   # Nazario + Nigerian_Fraud: cluster-grouped slice -> test_holdout3
+    new_source_val_fraction: 0.10    # of the remainder -> finetune_val
+    # A new near-dup cluster that links to BOTH train (or the MLM corpus) AND val is a 419 template the primary set
+    # already holds on both sides; dropping it keeps train_vs_val_shingle_links at the binary build's 0 (costs ~424
+    # Nigerian_Fraud rows). false = pin such clusters to train (adds ~226 info-only train/val near-dup links).
+    drop_new_clusters_linked_to_both_train_and_val: true
+    holdout3_from_val:               # clean-provenance legit / spam rows moved from finetune_val into test_holdout3
+      legit_sources: [CEAS-08, TREC-07, Assassin]
+      spam_sources: [Assassin, Enron]
+      legit_rows: 2000
+      spam_rows: 700
+      # exclusions: keys or >=3-shingle links into the pretraining corpus, clusters also in finetune_train,
+      # clusters touched by a new row, clusters with an ineligible member (whole clusters move or none)
+    display_names: {0: Legitimate, 1: Phishing, 2: Spam}
+    binary_rollup_name: "Unwanted (phishing or spam)"
+
+# ---------------------------------------------------------------- VRAM policy
+vram:
+  target_gb_low: 14
+  target_gb_high: 18
+  hard_cap_gb: 20
+  alloc_conf: "expandable_segments:True"
+  log_peak_every_steps: 20
+
+# ---------------------------------------------------------------- Phase 1: MLM pretraining
+pretrain:
+  precision: bf16_autocast
+  sequence_packing: true             # no attn_mask -> Flash SDPA backend
+  effective_batch_sequences: 256     # 256 x 512 = 131,072 tokens/step (BERT setting)
+  micro_batch: auto                  # batch finder picks the largest candidate whose reserved peak is in the VRAM band
+  micro_batch_start: 24              # Nathan's choice 2026-10-08 (17.65 GiB reserved, 25.2k tok/s); 256/24 -> accumulation 11 = effective 264
+  micro_batch_candidates: [16, 24, 12, 8]
+  epochs: 12                         # Nathan's choice 2026-10-08: ~14 h at 25k tok/s, ~9,550 steps; val MLM loss watched for a rise
+  optimizer: adamw
+  lr: 1.5e-4
+  betas: [0.9, 0.98]
+  eps: 1.0e-6
+  weight_decay: 0.01                 # excluded for biases and LayerNorm
+  grad_clip: 1.0
+  warmup_steps: 1000                 # cap; actual warmup = min(warmup_steps, warmup_fraction x total steps)
+  warmup_fraction: 0.06              # 3 epochs (~2,460 steps) -> ~148 warmup steps; 6 epochs -> ~295
+  schedule: cosine_to_10pct
+  mlm_probability: 0.15
+  mask_replace_probs: [0.8, 0.1, 0.1]  # [MASK] / random token / unchanged
+  loss_on_masked_only: true          # gather masked positions before the MLM head
+  checkpoint_every_steps: 500
+  log_every_steps: 20
+  eval_every_steps: 500
+
+# ---------------------------------------------------------------- Phase 2: fine-tuning (the instructor's dials)
+finetune:
+  # Three-way head (RUN_LOG entry 6): 0 Legitimate / 1 Phishing / 2 Spam. Lives here, NOT under model:, because the
+  # live pretraining run refuses to resume if its model snapshot changes. 2 reproduces the binary baseline runs.
+  # Rows with label_set "unwanted" (TREC-05/07, CEAS-08 positives) train on P(phishing)+P(spam) (partial-label NLL).
+  num_labels: 3
+  epochs: 3
+  batch_size: 32                     # examples per optimizer step (= micro_batch x accumulation)
+  micro_batch: 16                    # rows per forward/backward; measured ~11.8 GiB reserved at 16 x 512
+  eval_batch_size: 64
+  lr: 3.0e-5
+  weight_decay: 0.01
+  betas: [0.9, 0.98]
+  eps: 1.0e-6
+  grad_clip: 1.0
+  warmup_fraction: 0.06
+  schedule: linear_decay
+  eval_every_steps: 500
+  log_every_steps: 20
+  loss: cross_entropy                # experiment set: cross_entropy | weighted_cross_entropy | focal (all over the partial-label NLL)
+  class_weights: auto                # inverse-frequency over fully-labelled rows when loss == weighted_cross_entropy;
+                                     # "unwanted" rows get mean(w_phishing, w_spam)
+  focal_gamma: 2.0
+  early_stopping:
+    # val_f1_unwanted = F1 of the binary roll-up P(phishing)+P(spam) >= 0.5 (identical to the binary runs' val F1,
+    # so results stay comparable); alternative: val_macro_f1_3way (macro-F1 over the three classes on val rows
+    # with a full label).
+    metric: val_f1_unwanted
+    patience: 2
+  pooling: cls
+  freeze_encoder_layers: 0           # experiment dial (instructor: "number of layers")
+  dynamic_padding_multiple: 8
+  checkpoint_every_steps: 500
+
+# ---------------------------------------------------------------- evaluation
+eval:
+  metrics: [accuracy, precision, recall, f1, roc_auc, pr_auc, confusion_matrix, recall_at_fpr_1pct, recall_at_fpr_0_1pct]
+  credential_lure_slice: true        # secondary slice of the holdout matching credential-lure phrases
+  latency_samples: 100               # avg ms/email, GPU and CPU, like the instructor's step 5
+  latency_max_len: 512
+  threshold: 0.5                     # decision threshold; evaluate.py also reports the best-F1 threshold
+  # Three-way evaluation (RUN_LOG entry 6). The binary metrics above are unchanged: they are computed on the
+  # roll-up P(unwanted) = P(phishing) + P(spam) (= P(class 1) for a 2-class head), so results stay comparable
+  # with the binary-era runs. Three-way metrics use only rows with label3 in {0,1,2}; -1/"unwanted" rows
+  # (TREC-05 / TREC-07 / CEAS-08 positives) enter the roll-up only.
+  test3_file: test_holdout3.parquet  # clean-provenance three-way holdout written by data_prep (--split test3); read by evaluate.py
+  # The three keys below RESTATE the label contract that is fixed in code (evaluate.LABEL_NAMES / UNWANTED_NAME /
+  # DECISION_RULE, mirrored in export.py and the app). evaluate.py asserts at startup that they still match and
+  # refuses to run otherwise, so a drift is loud; editing them here does not rename anything.
+  label_names: {0: Legitimate, 1: Phishing, 2: Spam}
+  unwanted_name: "Unwanted (phishing or spam)"
+  threeway_decision_rule: rollup_then_argmax   # unwanted iff P(unwanted) >= threshold, then argmax(P(phishing), P(spam)); argmax-of-3 reported alongside
+  # documentation only: the sections every evaluated three-way block contains (not read by any script)
+  metrics_threeway: [confusion_matrix_3x3, per_class_precision_recall_f1, macro_f1, phishing_vs_spam_on_true_unwanted, per_source_counts]
+  top_k_confusions: 25               # read by evaluate.py: size of the phishing->spam / spam->phishing preview tables in error_analysis_test3.md
+  model_card_metrics3: reports/metrics_test3.json   # read by export.py: three-way table source (binary table: reports/metrics.json); both files must name the exported checkpoint
+
+# ---------------------------------------------------------------- Hugging Face (ask before any push)
+hf:
+  owner: BuildITSystems
+  model_repo: BuildITSystems/phishing-detector-encoder
+  space_repo: BuildITSystems/phishing-detector-demo
+  private: true
+  space_sdk: gradio
+  space_sdk_version: "6.29.1"
+  space_python_version: "3.12.12"    # ZeroGPU supports exactly 3.12.12 / 3.10.13
+  space_hardware: zero-a10g          # ZeroGPU flavor string (backing GPU: RTX Pro 6000 Blackwell slice)
+  space_torch: "2.13.0"              # highest version on the ZeroGPU allow-list (2.14.x not listed)
+  weights_format: safetensors_bf16
diff --git a/presentation/data/processed/data_report.md b/presentation/data/processed/data_report.md
new file mode 100644
index 0000000000000000000000000000000000000000..8bd81254a6aa59a2e9110e00cfcb7f1312e82d02
--- /dev/null
+++ b/presentation/data/processed/data_report.md
@@ -0,0 +1,239 @@
+# Data report — 2026-10-08T06:41:51+00:00
+
+spam_policy `positive_class` (positive = phishing/spam, unwanted); quote stripping on; seed 42; holdout source `TREC-06`; elapsed 139.0s; peak RSS 6.58 GB.
+
+## Sources
+- labelled primary: `puyang2025/seven-phishing-email-datasets` @ `b7f3f8d99ebd4fd0af0a5afe027420e53f08d1a4` (203,017 rows, hub splits {'train': 162413, 'test': 40604} concatenated; the hub split leaks and is ignored)
+- pretraining corpus: `corbt/enron-emails` @ `cfc06c758093d90993abce1a43668fb7357258a6` (517,401 rows)
+- excluded (no body text / 92% subset): `it4lia/PhishingEmailCuratedDatasets_Cleaned`, `zefang-liu/phishing-email-dataset`
+
+## Splits
+| split | rows | positives | negatives | base rate |
+|---|---|---|---|---|
+| finetune_train | 157,113 | 75,675 | 81,438 | 0.4817 |
+| finetune_val | 17,643 | 8,642 | 9,001 | 0.4898 |
+| test_holdout | 16,302 | 3,894 | 12,408 | 0.2389 |
+| pretrain (MLM) | 226,273 | - | - | 407,902,364 chars |
+
+### finetune_train by source
+| source | label 0 | label 1 | rows | positive rate |
+|---|---|---|---|---|
+| Assassin | 3,705 | 1,388 | 5,093 | 27.3% |
+| CEAS-08 | 15,547 | 18,631 | 34,178 | 54.5% |
+| Enron | 12,741 | 11,482 | 24,223 | 47.4% |
+| TREC-05 | 27,663 | 18,723 | 46,386 | 40.4% |
+| TREC-07 | 21,782 | 25,451 | 47,233 | 53.9% |
+
+### finetune_val by source
+| source | label 0 | label 1 | rows | positive rate |
+|---|---|---|---|---|
+| Assassin | 380 | 136 | 516 | 26.4% |
+| CEAS-08 | 1,699 | 2,184 | 3,883 | 56.2% |
+| Enron | 1,381 | 1,327 | 2,708 | 49.0% |
+| TREC-05 | 2,997 | 2,052 | 5,049 | 40.6% |
+| TREC-07 | 2,544 | 2,943 | 5,487 | 53.6% |
+
+### test_holdout by source
+| source | label 0 | label 1 | rows | positive rate |
+|---|---|---|---|---|
+| TREC-06 | 12,408 | 3,894 | 16,302 | 23.9% |
+
+## Normalisation and dials
+- common.normalize_text: NFKC -> lowercase -> keep [a-z0-9] only (import it, never re-derive it; NFKC is an extension of the verify_data.md section 7 wording)
+- dials: {'key_prefix_chars': 300, 'key_prefix_min_len': 80, 'body_key_min_alnum': 20, 'pretrain_min_alnum': 50, 'quote_strip_min_alnum': 200, 'shingle_cap': 2000}
+
+## Drops and removals
+- labelled rows with <20 alphanumerics (subject+body): 715 rows
+| source | label 0 | label 1 | rows | positive rate |
+|---|---|---|---|---|
+| CEAS-08 | 0 | 1 | 1 | 100.0% |
+| Enron | 12 | 18 | 30 | 60.0% |
+| TREC-05 | 51 | 526 | 577 | 91.2% |
+| TREC-06 | 3 | 95 | 98 | 96.9% |
+| TREC-07 | 0 | 9 | 9 | 100.0% |
+- Ling excluded (academic list corpus, 2,859 rows by label {'0': 2401, '1': 458})
+- pool label conflicts (identical normalised body labelled both 0 and 1; all members dropped): 18 rows in 8 groups
+| source | label 0 | label 1 | rows | positive rate |
+|---|---|---|---|---|
+| Enron | 8 | 0 | 8 | 0.0% |
+| TREC-05 | 0 | 10 | 10 | 100.0% |
+- pool exact duplicates (k_full, keep first): 6,715 rows
+| source | label 0 | label 1 | rows | positive rate |
+|---|---|---|---|---|
+| Assassin | 1 | 184 | 185 | 99.5% |
+| CEAS-08 | 58 | 1,020 | 1,078 | 94.6% |
+| Enron | 1,641 | 693 | 2,334 | 29.7% |
+| TREC-05 | 1,611 | 772 | 2,383 | 32.4% |
+| TREC-07 | 31 | 704 | 735 | 95.8% |
+- decontamination against the holdout: {'key_match_only': 4, 'shingle_link_only': 1516, 'both': 132, 'total': 1652}
+| source | label 0 | label 1 | rows | positive rate |
+|---|---|---|---|---|
+| Assassin | 1 | 10 | 11 | 90.9% |
+| CEAS-08 | 8 | 6 | 14 | 42.9% |
+| Enron | 8 | 456 | 464 | 98.3% |
+| TREC-05 | 7 | 863 | 870 | 99.2% |
+| TREC-07 | 1 | 292 | 293 | 99.7% |
+- holdout internal exact duplicates kept (test set is never deduped): 116
+
+## Near-duplicate clustering
+- 50-char shingles, stride 25, first 1500 chars, link at >= 3 shared; 1 boilerplate shingles over cap 2,000 ignored (touching 2,930 rows)
+- links at the removal stage (pool + holdout index): {'pool_pool': 3633009, 'pool_holdout': 5150, 'holdout_holdout': 15456}
+- clusters 114,261 (mixed source x label: 1,781; largest 7,069); val fraction target 0.1 achieved 0.101
+| stratum | rows | clusters | val target | val rows | val frac | clusters too large |
+|---|---|---|---|---|---|---|
+| Assassin|0 | 3,811 | 2,733 | 381 | 381 | 0.1 | 1 |
+| Assassin|1 | 1,436 | 834 | 144 | 144 | 0.1003 | 0 |
+| CEAS-08|0 | 16,993 | 11,149 | 1,699 | 1,699 | 0.1 | 1 |
+| CEAS-08|1 | 21,586 | 12,202 | 2,159 | 2,159 | 0.1 | 1 |
+| Enron|0 | 13,589 | 12,731 | 1,359 | 1,359 | 0.1 | 0 |
+| Enron|1 | 11,393 | 8,589 | 1,139 | 1,139 | 0.1 | 0 |
+| TREC-05|0 | 30,181 | 24,292 | 3,018 | 3,018 | 0.1 | 0 |
+| TREC-05|1 | 21,933 | 14,361 | 2,193 | 2,229 | 0.1016 | 0 |
+| TREC-07|0 | 25,438 | 11,740 | 2,544 | 2,544 | 0.1 | 1 |
+| TREC-07|1 | 28,396 | 15,630 | 2,840 | 2,971 | 0.1046 | 0 |
+
+## Pretraining corpus (corbt/enron-emails)
+| stage | rows |
+|---|---|
+| rows in | 517,401 |
+| exact dups (subject+body, raw) | 272,905 |
+| < 50 alnum after stripping | 14,027 |
+| exact dups after stripping | 107 |
+| matches a holdout key (removed) | 0 |
+| matches a fine-tune val key (removed) | 4,004 |
+| >= 3 shingles shared with a holdout row (removed) | 85 |
+| >= 3 shingles shared with a fine-tune val row (kept, counted) | 1,095 |
+| overlaps a fine-tune train key (kept, allowed) | 35,172 |
+| rows out | 226,273 |
+- near-dup screen: 88 links to 19 holdout rows (by label {'0': 18, '1': 1}); 5,345 links to 883 val rows
+- quote/forward stripping on: 21,832 bodies cut, 70,907 had a marker but kept the full body (< 200 alnum above it); marker hits {'forwarded': 33089, 'header_block': 5428, 'original': 54222}; retained chars 0.8948 of all bodies, 0.2548 of cut bodies
+- 407,902,364 chars out, mean 1802.7 chars/doc
+
+## Leakage assertions
+| check | count | asserted zero | pass |
+|---|---|---|---|
+| holdout_vs_train_key_rows | 0 | True | True |
+| holdout_vs_val_key_rows | 0 | True | True |
+| holdout_vs_pretrain_key_rows | 0 | True | True |
+| holdout_vs_train_shingle_links | 0 | True | True |
+| holdout_vs_val_shingle_links | 0 | True | True |
+| holdout_vs_pretrain_shingle_links | 0 | True | True |
+| train_vs_val_k_full_rows | 0 | True | True |
+| train_vs_val_any_key_rows | 0 | False | True |
+| train_vs_val_shingle_links | 0 | False | True |
+| val_vs_pretrain_key_rows | 0 | True | True |
+| val_vs_pretrain_shingle_links | 5,336 | False | True |
+| train_vs_pretrain_key_overlap | 102,100 | False | True |
+| holdout_rows_in_train_or_val_by_source | 0 | True | True |
+| test_holdout_rows_from_other_sources | 0 | True | True |
+| pretrain_rows | 226,273 | False | True |
+
+## Files
+- `finetune_train`: `/home/nathank/BITS/dev/Phishing Model/data/processed/finetune_train.parquet`
+- `finetune_val`: `/home/nathank/BITS/dev/Phishing Model/data/processed/finetune_val.parquet`
+- `test_holdout`: `/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet`
+- `pretrain`: `/home/nathank/BITS/dev/Phishing Model/data/processed/pretrain.parquet`
+
+## Three-way labels — build3 2026-10-08T20:21:36+00:00
+
+label3 {'0': 'legit', '1': 'phish', '2': 'spam', '-1': 'unwanted'}; display names {'0': 'Legitimate', '1': 'Phishing', '2': 'Spam'}; binary roll-up "Unwanted (phishing or spam)" = P(phishing)+P(spam). Positive rows by provenance: {'Assassin': 2, 'Enron': 2, 'Ling': 2, 'TREC-05': -1, 'TREC-06': -1, 'TREC-07': -1, 'CEAS-08': -1, 'Nazario': 1, 'Nigerian_Fraud': 1}. unchanged: 1 for phish / spam / unwanted, 0 for legit. Elapsed 66.1s; peak RSS 3.23 GB; binary base from snapshot (`/home/nathank/BITS/dev/Phishing Model/data/processed/binary_snapshot`).
+
+### New sources
+- `kudzaiprichard/aura-phishing-email-corpus` / `Nazario.csv` @ `539567cee21d03c83bd2fb1b142c18a80904eb72`: 1,565 rows (1,565 used), label3 1 (phishing by provenance)
+- `kudzaiprichard/aura-phishing-email-corpus` / `Nigerian_Fraud.csv` @ `539567cee21d03c83bd2fb1b142c18a80904eb72`: 3,332 rows (3,332 used), label3 1 (phishing by provenance)
+- bigshp (`locuoco/the-biggest-spam-ham-phish-email-dataset-300000`): research verdict **DO_NOT_USE**, not ingested (`docs/research/bigshp_research.md`)
+
+| source | loaded | dropped | kept | -> train | -> val | -> test_holdout3 |
+|---|---|---|---|---|---|---|
+| Nazario | 1,565 | 58 | 1,507 | 1,159 | 126 | 222 |
+| Nigerian_Fraud | 3,332 | 828 | 2,504 | 1,991 | 214 | 299 |
+
+- drops by reason: {'Nazario': {'mbox_internal_data': 2, 'within_new_k_full_dup': 56}, 'Nigerian_Fraud': {'shingle_link_test_holdout': 235, 'within_new_k_full_dup': 98, 'key_match_finetune_train': 54, 'near_dup_of_both_train_and_val': 424, 'key_match_test_holdout': 13, 'lt_20_alnum': 1, 'key_match_finetune_val': 3}}
+- near-dup pins: {'clusters_total': 3239, 'clusters_pinned_train_by_train_or_pretrain_link': 238, 'clusters_pinned_val_by_val_link': 23, 'clusters_dropped_linked_to_both_train_and_val': 17, 'rows_dropped_linked_to_both_train_and_val': 424, 'clusters_with_a_holdout_linked_member': 235, 'rows_linked_to_train': 329, 'rows_linked_to_pretrain': 33, 'rows_linked_to_val': 35, 'rows_dropped_holdout_shingle_link': 235, 'largest_cluster': 52}
+- bigshp label contradictions among exact matches: 0
+
+### test_holdout3 rows taken from finetune_val
+- legitimate from ['CEAS-08', 'TREC-07', 'Assassin'], spam from ['Assassin', 'Enron']; whole near-dup clusters only; excluded candidates by reason: {'Enron': {'cluster_has_an_ineligible_member': 261, 'shingle_link_to_a_new_row': 28, 'shingle_link_pretrain': 4}, 'Assassin': {'shingle_link_pretrain': 8, 'cluster_has_an_ineligible_member': 7}, 'CEAS-08': {'shingle_link_pretrain': 1}, 'TREC-07': {'cluster_has_an_ineligible_member': 1}}
+- per class: {'Legitimate': {'target_rows': 2000, 'eligible_rows': 4621, 'fraction': 0.4328, 'moved_rows': 1999}, 'Spam': {'target_rows': 700, 'eligible_rows': 1155, 'fraction': 0.6061, 'moved_rows': 700}}
+| source | 0 Legitimate | 1 Phishing | 2 Spam | -1 unwanted (unsplit) | rows |
+|---|---|---|---|---|---|
+| Assassin | 164 | 0 | 72 | 0 | 236 |
+| CEAS-08 | 736 | 0 | 0 | 0 | 736 |
+| Enron | 0 | 0 | 628 | 0 | 628 |
+| TREC-07 | 1,099 | 0 | 0 | 0 | 1,099 |
+
+### Splits by label3
+#### finetune_train: 160,263 rows, by label3 {'-1': 62805, '0': 81438, '1': 3150, '2': 12870}, binary positives 78,825 (0.4918)
+| source | 0 Legitimate | 1 Phishing | 2 Spam | -1 unwanted (unsplit) | rows |
+|---|---|---|---|---|---|
+| Assassin | 3,705 | 0 | 1,388 | 0 | 5,093 |
+| CEAS-08 | 15,547 | 0 | 0 | 18,631 | 34,178 |
+| Enron | 12,741 | 0 | 11,482 | 0 | 24,223 |
+| Nazario | 0 | 1,159 | 0 | 0 | 1,159 |
+| Nigerian_Fraud | 0 | 1,991 | 0 | 0 | 1,991 |
+| TREC-05 | 27,663 | 0 | 0 | 18,723 | 46,386 |
+| TREC-07 | 21,782 | 0 | 0 | 25,451 | 47,233 |
+
+#### finetune_val: 15,284 rows, by label3 {'-1': 7179, '0': 7002, '1': 340, '2': 763}, binary positives 8,282 (0.5419)
+| source | 0 Legitimate | 1 Phishing | 2 Spam | -1 unwanted (unsplit) | rows |
+|---|---|---|---|---|---|
+| Assassin | 216 | 0 | 64 | 0 | 280 |
+| CEAS-08 | 963 | 0 | 0 | 2,184 | 3,147 |
+| Enron | 1,381 | 0 | 699 | 0 | 2,080 |
+| Nazario | 0 | 126 | 0 | 0 | 126 |
+| Nigerian_Fraud | 0 | 214 | 0 | 0 | 214 |
+| TREC-05 | 2,997 | 0 | 0 | 2,052 | 5,049 |
+| TREC-07 | 1,445 | 0 | 0 | 2,943 | 4,388 |
+
+#### test_holdout: 16,302 rows, by label3 {'-1': 3894, '0': 12408}, binary positives 3,894 (0.2389)
+| source | 0 Legitimate | 1 Phishing | 2 Spam | -1 unwanted (unsplit) | rows |
+|---|---|---|---|---|---|
+| TREC-06 | 12,408 | 0 | 0 | 3,894 | 16,302 |
+
+#### test_holdout3: 3,220 rows, by label3 {'0': 1999, '1': 521, '2': 700}, binary positives 1,221 (0.3792)
+| source | 0 Legitimate | 1 Phishing | 2 Spam | -1 unwanted (unsplit) | rows |
+|---|---|---|---|---|---|
+| Assassin | 164 | 0 | 72 | 0 | 236 |
+| CEAS-08 | 736 | 0 | 0 | 0 | 736 |
+| Enron | 0 | 0 | 628 | 0 | 628 |
+| Nazario | 0 | 222 | 0 | 0 | 222 |
+| Nigerian_Fraud | 0 | 299 | 0 | 0 | 299 |
+| TREC-07 | 1,099 | 0 | 0 | 0 | 1,099 |
+
+### Leakage assertions (binary gate + three-way)
+| check | count | asserted zero | pass |
+|---|---|---|---|
+| holdout_vs_train_key_rows | 0 | True | True |
+| holdout_vs_val_key_rows | 0 | True | True |
+| holdout_vs_pretrain_key_rows | 0 | True | True |
+| holdout_vs_train_shingle_links | 0 | True | True |
+| holdout_vs_val_shingle_links | 0 | True | True |
+| holdout_vs_pretrain_shingle_links | 0 | True | True |
+| train_vs_val_k_full_rows | 0 | True | True |
+| train_vs_val_any_key_rows | 0 | False | True |
+| train_vs_val_shingle_links | 0 | False | True |
+| val_vs_pretrain_key_rows | 0 | True | True |
+| val_vs_pretrain_shingle_links | 5,336 | False | True |
+| train_vs_pretrain_key_overlap | 102,100 | False | True |
+| holdout_rows_in_train_or_val_by_source | 0 | True | True |
+| test_holdout_rows_from_other_sources | 0 | True | True |
+| pretrain_rows | 226,273 | False | True |
+| holdout3_vs_train_key_rows | 0 | True | True |
+| holdout3_vs_val_key_rows | 0 | True | True |
+| holdout3_vs_pretrain_key_rows | 0 | True | True |
+| holdout3_vs_holdout_key_rows | 0 | True | True |
+| holdout3_vs_train_shingle_links | 0 | True | True |
+| holdout3_vs_val_shingle_links | 0 | True | True |
+| holdout3_vs_pretrain_shingle_links | 0 | True | True |
+| holdout3_vs_holdout_shingle_links | 0 | False | True |
+| holdout3_rows_without_full_label3 | 0 | True | True |
+| holdout3_rows_from_holdout_source | 0 | True | True |
+| doc_id_shared_between_splits | 0 | True | True |
+| label3_contract_violations | 0 | True | True |
+| holdout3_rows | 3,220 | False | True |
+
+### Files
+- `finetune_train`: `/home/nathank/BITS/dev/Phishing Model/data/processed/finetune_train.parquet`
+- `finetune_val`: `/home/nathank/BITS/dev/Phishing Model/data/processed/finetune_val.parquet`
+- `test_holdout`: `/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet`
+- `test_holdout3`: `/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout3.parquet`
diff --git a/presentation/data/processed/pretrain_tokens.json b/presentation/data/processed/pretrain_tokens.json
new file mode 100644
index 0000000000000000000000000000000000000000..29b8f1df5437a426186580c036baabe409a4176c
--- /dev/null
+++ b/presentation/data/processed/pretrain_tokens.json
@@ -0,0 +1,22 @@
+{
+  "created": "2026-10-08T00:24:30",
+  "parquet": "/home/nathank/BITS/dev/Phishing Model/data/processed/pretrain.parquet",
+  "parquet_rows": 226273,
+  "n_docs": 226273,
+  "limit_docs": null,
+  "n_docs_train": 224011,
+  "n_docs_val": 2262,
+  "val_rule": "last 1% of documents by parquet order",
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "tokenizer_sha1": "9f50c5e775089d2ca3efc82e100cd6b3396ed7e7",
+  "vocab_size": 32768,
+  "sep_id": 3,
+  "mask_id": 4,
+  "dtype": "uint16",
+  "n_tokens_train": 107498500,
+  "n_tokens_val": 1021888,
+  "chars": 407902364,
+  "tokens_per_char": 0.26605,
+  "encode_seconds": 42.8,
+  "packing": "documents concatenated with [SEP] after each; windows of max_seq_len cut from the stream"
+}
diff --git a/presentation/data/processed/splits_manifest.json b/presentation/data/processed/splits_manifest.json
new file mode 100644
index 0000000000000000000000000000000000000000..8282b08fd5040d27fba1075636351b21b30d8ba5
--- /dev/null
+++ b/presentation/data/processed/splits_manifest.json
@@ -0,0 +1,1383 @@
+{
+  "timestamp": "2026-10-08T06:41:51+00:00",
+  "limit": null,
+  "dev_subsample": false,
+  "spam_policy": "positive_class",
+  "strip_quotes": true,
+  "seed": 42,
+  "holdout_test_source": "TREC-06",
+  "elapsed_seconds": 139.0,
+  "peak_rss_gb": 6.58,
+  "datasets": {
+    "labelled_primary": {
+      "repo": "puyang2025/seven-phishing-email-datasets",
+      "commit_sha": "b7f3f8d99ebd4fd0af0a5afe027420e53f08d1a4",
+      "hub_splits_concatenated": {
+        "train": 162413,
+        "test": 40604
+      },
+      "datasets_fingerprints": {
+        "train": "c8b77d5643837650",
+        "test": "49f7225cee80b61f"
+      },
+      "rows_loaded": 203017
+    },
+    "pretrain_corpus": {
+      "repo": "corbt/enron-emails",
+      "commit_sha": "cfc06c758093d90993abce1a43668fb7357258a6",
+      "datasets_fingerprint": "319d5c9cdca6d35d",
+      "rows_total": 517401
+    },
+    "excluded": [
+      "it4lia/PhishingEmailCuratedDatasets_Cleaned",
+      "zefang-liu/phishing-email-dataset"
+    ]
+  },
+  "counts": {
+    "splits": {
+      "finetune_train": {
+        "Assassin": {
+          "0": 3705,
+          "1": 1388
+        },
+        "CEAS-08": {
+          "0": 15547,
+          "1": 18631
+        },
+        "Enron": {
+          "0": 12741,
+          "1": 11482
+        },
+        "TREC-05": {
+          "0": 27663,
+          "1": 18723
+        },
+        "TREC-07": {
+          "0": 21782,
+          "1": 25451
+        }
+      },
+      "finetune_val": {
+        "Assassin": {
+          "0": 380,
+          "1": 136
+        },
+        "CEAS-08": {
+          "0": 1699,
+          "1": 2184
+        },
+        "Enron": {
+          "0": 1381,
+          "1": 1327
+        },
+        "TREC-05": {
+          "0": 2997,
+          "1": 2052
+        },
+        "TREC-07": {
+          "0": 2544,
+          "1": 2943
+        }
+      },
+      "test_holdout": {
+        "TREC-06": {
+          "0": 12408,
+          "1": 3894
+        }
+      }
+    },
+    "totals": {
+      "finetune_train": {
+        "rows": 157113,
+        "positives": 75675,
+        "negatives": 81438,
+        "base_rate": 0.4817
+      },
+      "finetune_val": {
+        "rows": 17643,
+        "positives": 8642,
+        "negatives": 9001,
+        "base_rate": 0.4898
+      },
+      "test_holdout": {
+        "rows": 16302,
+        "positives": 3894,
+        "negatives": 12408,
+        "base_rate": 0.2389
+      },
+      "pretrain": {
+        "rows": 226273,
+        "chars": 407902364
+      }
+    }
+  },
+  "normalization": "common.normalize_text: NFKC -> lowercase -> keep [a-z0-9] only (import it, never re-derive it; NFKC is an extension of the verify_data.md section 7 wording)",
+  "dials": {
+    "key_prefix_chars": 300,
+    "key_prefix_min_len": 80,
+    "body_key_min_alnum": 20,
+    "pretrain_min_alnum": 50,
+    "quote_strip_min_alnum": 200,
+    "shingle_cap": 2000
+  },
+  "drops": {
+    "labelled_min_alnum": {
+      "threshold": 20,
+      "rows": 715,
+      "by_source_label": {
+        "CEAS-08": {
+          "1": 1
+        },
+        "Enron": {
+          "0": 12,
+          "1": 18
+        },
+        "TREC-05": {
+          "0": 51,
+          "1": 526
+        },
+        "TREC-06": {
+          "0": 3,
+          "1": 95
+        },
+        "TREC-07": {
+          "1": 9
+        }
+      }
+    },
+    "ling_excluded_by_label": {
+      "0": 2401,
+      "1": 458
+    },
+    "pool_label_conflicts": {
+      "groups": 8,
+      "rows_dropped": 18,
+      "by_source_label": {
+        "Enron": {
+          "0": 8
+        },
+        "TREC-05": {
+          "1": 10
+        }
+      }
+    },
+    "pool_exact_dups_by_source_label": {
+      "Assassin": {
+        "0": 1,
+        "1": 184
+      },
+      "CEAS-08": {
+        "0": 58,
+        "1": 1020
+      },
+      "Enron": {
+        "0": 1641,
+        "1": 693
+      },
+      "TREC-05": {
+        "0": 1611,
+        "1": 772
+      },
+      "TREC-07": {
+        "0": 31,
+        "1": 704
+      }
+    },
+    "decontamination_vs_holdout": {
+      "by_source_label": {
+        "Assassin": {
+          "0": 1,
+          "1": 10
+        },
+        "CEAS-08": {
+          "0": 8,
+          "1": 6
+        },
+        "Enron": {
+          "0": 8,
+          "1": 456
+        },
+        "TREC-05": {
+          "0": 7,
+          "1": 863
+        },
+        "TREC-07": {
+          "0": 1,
+          "1": 292
+        }
+      },
+      "by_reason": {
+        "key_match_only": 4,
+        "shingle_link_only": 1516,
+        "both": 132,
+        "total": 1652
+      }
+    }
+  },
+  "holdout": {
+    "rows": 16302,
+    "internal_exact_dups_kept": 116
+  },
+  "near_dup": {
+    "shingle_chars": 50,
+    "stride": 25,
+    "window_chars": 1500,
+    "min_shared": 3,
+    "cap": 2000,
+    "capped_shingles": 1,
+    "rows_touching_capped": 2930,
+    "links_at_removal_stage": {
+      "pool_pool": 3633009,
+      "pool_holdout": 5150,
+      "holdout_holdout": 15456
+    }
+  },
+  "split": {
+    "val_fraction_target": 0.1,
+    "val_fraction_achieved": 0.101,
+    "clusters_total": 114261,
+    "clusters_mixed_stratum": 1781,
+    "largest_cluster": 7069,
+    "per_stratum": {
+      "Assassin|0": {
+        "rows": 3811,
+        "clusters": 2733,
+        "val_target": 381,
+        "val_rows": 381,
+        "val_fraction": 0.1,
+        "clusters_too_large_for_val": 1
+      },
+      "Assassin|1": {
+        "rows": 1436,
+        "clusters": 834,
+        "val_target": 144,
+        "val_rows": 144,
+        "val_fraction": 0.1003,
+        "clusters_too_large_for_val": 0
+      },
+      "CEAS-08|0": {
+        "rows": 16993,
+        "clusters": 11149,
+        "val_target": 1699,
+        "val_rows": 1699,
+        "val_fraction": 0.1,
+        "clusters_too_large_for_val": 1
+      },
+      "CEAS-08|1": {
+        "rows": 21586,
+        "clusters": 12202,
+        "val_target": 2159,
+        "val_rows": 2159,
+        "val_fraction": 0.1,
+        "clusters_too_large_for_val": 1
+      },
+      "Enron|0": {
+        "rows": 13589,
+        "clusters": 12731,
+        "val_target": 1359,
+        "val_rows": 1359,
+        "val_fraction": 0.1,
+        "clusters_too_large_for_val": 0
+      },
+      "Enron|1": {
+        "rows": 11393,
+        "clusters": 8589,
+        "val_target": 1139,
+        "val_rows": 1139,
+        "val_fraction": 0.1,
+        "clusters_too_large_for_val": 0
+      },
+      "TREC-05|0": {
+        "rows": 30181,
+        "clusters": 24292,
+        "val_target": 3018,
+        "val_rows": 3018,
+        "val_fraction": 0.1,
+        "clusters_too_large_for_val": 0
+      },
+      "TREC-05|1": {
+        "rows": 21933,
+        "clusters": 14361,
+        "val_target": 2193,
+        "val_rows": 2229,
+        "val_fraction": 0.1016,
+        "clusters_too_large_for_val": 0
+      },
+      "TREC-07|0": {
+        "rows": 25438,
+        "clusters": 11740,
+        "val_target": 2544,
+        "val_rows": 2544,
+        "val_fraction": 0.1,
+        "clusters_too_large_for_val": 1
+      },
+      "TREC-07|1": {
+        "rows": 28396,
+        "clusters": 15630,
+        "val_target": 2840,
+        "val_rows": 2971,
+        "val_fraction": 0.1046,
+        "clusters_too_large_for_val": 0
+      }
+    }
+  },
+  "pretrain": {
+    "rows_in": 517401,
+    "exact_dups_raw": 272905,
+    "short_after_strip": 14027,
+    "exact_dups_after_strip": 107,
+    "holdout_key_removed": 0,
+    "val_key_removed": 4004,
+    "train_overlap_kept": 35172,
+    "holdout_near_dup_removed": 85,
+    "near_dup_screen": {
+      "holdout_links": 88,
+      "holdout_rows_removed": 85,
+      "holdout_rows_hit_by_label": {
+        "0": 18,
+        "1": 1
+      },
+      "val_links": 5345,
+      "val_rows_hit": 883,
+      "pretrain_rows_near_val_kept": 1095
+    },
+    "rows_out": 226273,
+    "total_chars_out": 407902364,
+    "mean_chars_out": 1802.7,
+    "min_alnum_after_strip": 50,
+    "quote_strip": {
+      "enabled": true,
+      "bodies_cut": 21832,
+      "marker_present_but_kept_full": 70907,
+      "marker_hits": {
+        "forwarded": 33089,
+        "header_block": 5428,
+        "original": 54222
+      },
+      "retained_char_fraction_all_bodies": 0.8948,
+      "retained_char_fraction_cut_bodies": 0.2548,
+      "min_alnum_above_cut": 200
+    }
+  },
+  "leakage": {
+    "holdout_vs_train_key_rows": {
+      "count": 0,
+      "asserted": true,
+      "pass": true,
+      "detail": ""
+    },
+    "holdout_vs_val_key_rows": {
+      "count": 0,
+      "asserted": true,
+      "pass": true,
+      "detail": ""
+    },
+    "holdout_vs_pretrain_key_rows": {
+      "count": 0,
+      "asserted": true,
+      "pass": true,
+      "detail": "pretrain keys re-derived from stored text (text_identity)"
+    },
+    "holdout_vs_train_shingle_links": {
+      "count": 0,
+      "asserted": true,
+      "pass": true,
+      "detail": ""
+    },
+    "holdout_vs_val_shingle_links": {
+      "count": 0,
+      "asserted": true,
+      "pass": true,
+      "detail": ""
+    },
+    "holdout_vs_pretrain_shingle_links": {
+      "count": 0,
+      "asserted": true,
+      "pass": true,
+      "detail": "pretrain bases re-derived from stored text (text_identity)"
+    },
+    "train_vs_val_k_full_rows": {
+      "count": 0,
+      "asserted": true,
+      "pass": true,
+      "detail": ""
+    },
+    "train_vs_val_any_key_rows": {
+      "count": 0,
+      "asserted": false,
+      "pass": true,
+      "detail": ""
+    },
+    "train_vs_val_shingle_links": {
+      "count": 0,
+      "asserted": false,
+      "pass": true,
+      "detail": ""
+    },
+    "val_vs_pretrain_key_rows": {
+      "count": 0,
+      "asserted": true,
+      "pass": true,
+      "detail": ""
+    },
+    "val_vs_pretrain_shingle_links": {
+      "count": 5336,
+      "asserted": false,
+      "pass": true,
+      "detail": "near-twins of val emails in MLM text; counted, not removed (val is model selection, the holdout is the gate)"
+    },
+    "train_vs_pretrain_key_overlap": {
+      "count": 102100,
+      "asserted": false,
+      "pass": true,
+      "detail": "allowed: TREC-05/Enron ham are Enron mail"
+    },
+    "holdout_rows_in_train_or_val_by_source": {
+      "count": 0,
+      "asserted": true,
+      "pass": true,
+      "detail": ""
+    },
+    "test_holdout_rows_from_other_sources": {
+      "count": 0,
+      "asserted": true,
+      "pass": true,
+      "detail": ""
+    },
+    "pretrain_rows": {
+      "count": 226273,
+      "asserted": false,
+      "pass": true,
+      "detail": ""
+    }
+  },
+  "files": {
+    "finetune_train": "/home/nathank/BITS/dev/Phishing Model/data/processed/finetune_train.parquet",
+    "finetune_val": "/home/nathank/BITS/dev/Phishing Model/data/processed/finetune_val.parquet",
+    "test_holdout": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet",
+    "pretrain": "/home/nathank/BITS/dev/Phishing Model/data/processed/pretrain.parquet"
+  },
+  "config_snapshot": {
+    "seed": 42,
+    "paths": {
+      "data_raw": "/home/nathank/BITS/dev/Phishing Model/data/raw",
+      "data_processed": "/home/nathank/BITS/dev/Phishing Model/data/processed",
+      "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+      "checkpoints": "/home/nathank/BITS/dev/Phishing Model/checkpoints",
+      "reports": "/home/nathank/BITS/dev/Phishing Model/reports",
+      "logs": "/home/nathank/BITS/dev/Phishing Model/logs",
+      "app_dir": "/home/nathank/BITS/dev/Phishing Model/app"
+    },
+    "model": {
+      "n_layers": 24,
+      "d_model": 1024,
+      "n_heads": 16,
+      "d_ff": 4096,
+      "vocab_size": 32768,
+      "max_seq_len": 512,
+      "dropout": 0.1,
+      "attention_dropout": 0.1,
+      "norm": "pre_ln",
+      "activation": "gelu",
+      "tie_mlm_head": true,
+      "grad_checkpointing": false
+    },
+    "tokenizer": {
+      "type": "byte_level_bpe",
+      "vocab_size": 32768,
+      "min_frequency": 2,
+      "special_tokens": [
+        "[PAD]",
+        "[UNK]",
+        "[CLS]",
+        "[SEP]",
+        "[MASK]"
+      ],
+      "train_on": [
+        "pretrain",
+        "finetune_train"
+      ]
+    },
+    "data": {
+      "labelled_primary": "puyang2025/seven-phishing-email-datasets",
+      "labelled_excluded": [
+        "it4lia/PhishingEmailCuratedDatasets_Cleaned",
+        "zefang-liu/phishing-email-dataset"
+      ],
+      "pretrain_corpus": "corbt/enron-emails",
+      "holdout_test_source": "TREC-06",
+      "second_choice_holdout": "CEAS-08",
+      "spam_policy": "positive_class",
+      "min_alnum_chars": 20,
+      "dedup": {
+        "normalize": "lowercase; keep [a-z0-9] only",
+        "keys": [
+          "full_sha1",
+          "prefix300_sha1",
+          "subject_body_prefix300_sha1"
+        ],
+        "near_dup_shingle_chars": 50,
+        "near_dup_stride": 25,
+        "near_dup_min_shared": 3,
+        "near_dup_window_chars": 1500
+      },
+      "finetune_val_fraction": 0.1,
+      "model_input": "subject + \"\\n\\n\" + body",
+      "synthetic": {
+        "enabled": false,
+        "max_rows": 2000
+      }
+    },
+    "vram": {
+      "target_gb_low": 14,
+      "target_gb_high": 18,
+      "hard_cap_gb": 20,
+      "alloc_conf": "expandable_segments:True",
+      "log_peak_every_steps": 20
+    },
+    "pretrain": {
+      "precision": "bf16_autocast",
+      "sequence_packing": true,
+      "effective_batch_sequences": 256,
+      "micro_batch": "auto",
+      "micro_batch_start": 16,
+      "epochs": 3,
+      "optimizer": "adamw",
+      "lr": 0.00015,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "weight_decay": 0.01,
+      "grad_clip": 1.0,
+      "warmup_steps": 1000,
+      "schedule": "cosine_to_10pct",
+      "mlm_probability": 0.15,
+      "mask_replace_probs": [
+        0.8,
+        0.1,
+        0.1
+      ],
+      "loss_on_masked_only": true,
+      "checkpoint_every_steps": 500,
+      "log_every_steps": 20,
+      "eval_every_steps": 500
+    },
+    "finetune": {
+      "epochs": 3,
+      "batch_size": 32,
+      "lr": 3e-05,
+      "warmup_fraction": 0.06,
+      "schedule": "linear_decay",
+      "loss": "cross_entropy",
+      "class_weights": "auto",
+      "focal_gamma": 2.0,
+      "early_stopping": {
+        "metric": "val_f1_phishing",
+        "patience": 2
+      },
+      "pooling": "cls",
+      "freeze_encoder_layers": 0,
+      "dynamic_padding_multiple": 8,
+      "checkpoint_every_steps": 500
+    },
+    "eval": {
+      "metrics": [
+        "accuracy",
+        "precision",
+        "recall",
+        "f1",
+        "roc_auc",
+        "pr_auc",
+        "confusion_matrix",
+        "recall_at_fpr_1pct",
+        "recall_at_fpr_0_1pct"
+      ],
+      "credential_lure_slice": true,
+      "latency_samples": 100,
+      "latency_max_len": 512
+    },
+    "hf": {
+      "owner": "BuildITSystems",
+      "model_repo": "BuildITSystems/phishing-detector-encoder",
+      "space_repo": "BuildITSystems/phishing-detector-demo",
+      "private": true,
+      "space_sdk": "gradio",
+      "space_sdk_version": "6.29.1",
+      "space_python_version": "3.12.12",
+      "space_hardware": "zero-a10g",
+      "space_torch": "2.13.0",
+      "weights_format": "safetensors_bf16"
+    }
+  },
+  "threeway": {
+    "timestamp": "2026-10-08T20:21:36+00:00",
+    "limit": null,
+    "dev_subsample": false,
+    "seed": 42,
+    "rng_stream": "default_rng(seed + 3)",
+    "elapsed_seconds": 66.1,
+    "peak_rss_gb": 3.23,
+    "contract": {
+      "label3": {
+        "0": "legit",
+        "1": "phish",
+        "2": "spam",
+        "-1": "unwanted"
+      },
+      "display_names": {
+        "0": "Legitimate",
+        "1": "Phishing",
+        "2": "Spam"
+      },
+      "binary_rollup_name": "Unwanted (phishing or spam)",
+      "positive_label3_by_source": {
+        "Assassin": 2,
+        "Enron": 2,
+        "Ling": 2,
+        "TREC-05": -1,
+        "TREC-06": -1,
+        "TREC-07": -1,
+        "CEAS-08": -1,
+        "Nazario": 1,
+        "Nigerian_Fraud": 1
+      },
+      "binary_label": "unchanged: 1 for phish / spam / unwanted, 0 for legit"
+    },
+    "binary_base": {
+      "from": "snapshot",
+      "dir": "/home/nathank/BITS/dev/Phishing Model/data/processed/binary_snapshot",
+      "counts_by_source_label3": {
+        "finetune_train": {
+          "Assassin": {
+            "0": 3705,
+            "2": 1388
+          },
+          "CEAS-08": {
+            "-1": 18631,
+            "0": 15547
+          },
+          "Enron": {
+            "0": 12741,
+            "2": 11482
+          },
+          "TREC-05": {
+            "-1": 18723,
+            "0": 27663
+          },
+          "TREC-07": {
+            "-1": 25451,
+            "0": 21782
+          }
+        },
+        "finetune_val": {
+          "Assassin": {
+            "0": 380,
+            "2": 136
+          },
+          "CEAS-08": {
+            "-1": 2184,
+            "0": 1699
+          },
+          "Enron": {
+            "0": 1381,
+            "2": 1327
+          },
+          "TREC-05": {
+            "-1": 2052,
+            "0": 2997
+          },
+          "TREC-07": {
+            "-1": 2943,
+            "0": 2544
+          }
+        },
+        "test_holdout": {
+          "TREC-06": {
+            "-1": 3894,
+            "0": 12408
+          }
+        }
+      },
+      "rows": {
+        "finetune_train": 157113,
+        "finetune_val": 17643,
+        "test_holdout": 16302
+      }
+    },
+    "sources": {
+      "champa": {
+        "repo": "kudzaiprichard/aura-phishing-email-corpus",
+        "local_dir": "/home/nathank/BITS/dev/Phishing Model/data/raw/aura",
+        "label3": 1,
+        "files": {
+          "Nazario.csv": {
+            "rows": 1565,
+            "rows_used": 1565,
+            "commit_sha": "539567cee21d03c83bd2fb1b142c18a80904eb72",
+            "path": "/home/nathank/BITS/dev/Phishing Model/data/raw/aura/Nazario.csv",
+            "bytes": 7811841
+          },
+          "Nigerian_Fraud.csv": {
+            "rows": 3332,
+            "rows_used": 3332,
+            "commit_sha": "539567cee21d03c83bd2fb1b142c18a80904eb72",
+            "path": "/home/nathank/BITS/dev/Phishing Model/data/raw/aura/Nigerian_Fraud.csv",
+            "bytes": 9192393
+          }
+        }
+      },
+      "bigshp": {
+        "enabled": false,
+        "verdict": "DO_NOT_USE",
+        "research": "docs/research/bigshp_research.md"
+      }
+    },
+    "new_rows": {
+      "loaded_by_source": {
+        "Nigerian_Fraud": 3332,
+        "Nazario": 1565
+      },
+      "drops_by_source_reason": {
+        "Nazario": {
+          "mbox_internal_data": 2,
+          "within_new_k_full_dup": 56
+        },
+        "Nigerian_Fraud": {
+          "shingle_link_test_holdout": 235,
+          "within_new_k_full_dup": 98,
+          "key_match_finetune_train": 54,
+          "near_dup_of_both_train_and_val": 424,
+          "key_match_test_holdout": 13,
+          "lt_20_alnum": 1,
+          "key_match_finetune_val": 3
+        }
+      },
+      "bigshp_label_contradictions_among_key_matches": 0,
+      "kept_by_source": {
+        "Nigerian_Fraud": 2504,
+        "Nazario": 1507
+      },
+      "destination_by_source": {
+        "Nazario": {
+          "train": 1159,
+          "test3": 222,
+          "val": 126
+        },
+        "Nigerian_Fraud": {
+          "train": 1991,
+          "test3": 299,
+          "val": 214
+        }
+      },
+      "pins": {
+        "clusters_total": 3239,
+        "clusters_pinned_train_by_train_or_pretrain_link": 238,
+        "clusters_pinned_val_by_val_link": 23,
+        "clusters_dropped_linked_to_both_train_and_val": 17,
+        "rows_dropped_linked_to_both_train_and_val": 424,
+        "clusters_with_a_holdout_linked_member": 235,
+        "rows_linked_to_train": 329,
+        "rows_linked_to_pretrain": 33,
+        "rows_linked_to_val": 35,
+        "rows_dropped_holdout_shingle_link": 235,
+        "largest_cluster": 52
+      },
+      "test_fraction": 0.15,
+      "val_fraction_of_remainder": 0.1,
+      "split_test3": {
+        "val_fraction_target": 0.15,
+        "val_fraction_achieved": 0.15,
+        "clusters_total": 2978,
+        "clusters_mixed_stratum": 0,
+        "largest_cluster": 52,
+        "per_stratum": {
+          "Nazario|1": {
+            "rows": 1478,
+            "clusters": 1245,
+            "val_target": 222,
+            "val_rows": 222,
+            "val_fraction": 0.1502,
+            "clusters_too_large_for_val": 0
+          },
+          "Nigerian_Fraud|1": {
+            "rows": 1995,
+            "clusters": 1733,
+            "val_target": 299,
+            "val_rows": 299,
+            "val_fraction": 0.1499,
+            "clusters_too_large_for_val": 0
+          }
+        }
+      },
+      "split_val": {
+        "val_fraction_target": 0.1,
+        "val_fraction_achieved": 0.1003,
+        "clusters_total": 2547,
+        "clusters_mixed_stratum": 0,
+        "largest_cluster": 52,
+        "per_stratum": {
+          "Nazario|1": {
+            "rows": 1256,
+            "clusters": 1074,
+            "val_target": 126,
+            "val_rows": 126,
+            "val_fraction": 0.1003,
+            "clusters_too_large_for_val": 0
+          },
+          "Nigerian_Fraud|1": {
+            "rows": 1696,
+            "clusters": 1473,
+            "val_target": 170,
+            "val_rows": 170,
+            "val_fraction": 0.1002,
+            "clusters_too_large_for_val": 0
+          }
+        }
+      },
+      "cluster_id_offset": 127731
+    },
+    "holdout3_from_val": {
+      "legit_sources": [
+        "CEAS-08",
+        "TREC-07",
+        "Assassin"
+      ],
+      "spam_sources": [
+        "Assassin",
+        "Enron"
+      ],
+      "candidates_by_source_label3": {
+        "Assassin": {
+          "0": 380,
+          "2": 136
+        },
+        "CEAS-08": {
+          "0": 1699
+        },
+        "Enron": {
+          "2": 1327
+        },
+        "TREC-07": {
+          "0": 2544
+        }
+      },
+      "excluded_candidates_by_source_reason": {
+        "Enron": {
+          "cluster_has_an_ineligible_member": 261,
+          "shingle_link_to_a_new_row": 28,
+          "shingle_link_pretrain": 4
+        },
+        "Assassin": {
+          "shingle_link_pretrain": 8,
+          "cluster_has_an_ineligible_member": 7
+        },
+        "CEAS-08": {
+          "shingle_link_pretrain": 1
+        },
+        "TREC-07": {
+          "cluster_has_an_ineligible_member": 1
+        }
+      },
+      "eligible_by_source_label3": {
+        "Assassin": {
+          "0": 380,
+          "2": 121
+        },
+        "CEAS-08": {
+          "0": 1698
+        },
+        "Enron": {
+          "2": 1034
+        },
+        "TREC-07": {
+          "0": 2543
+        }
+      },
+      "moved_by_source_label3": {
+        "Assassin": {
+          "0": 164,
+          "2": 72
+        },
+        "CEAS-08": {
+          "0": 736
+        },
+        "Enron": {
+          "2": 628
+        },
+        "TREC-07": {
+          "0": 1099
+        }
+      },
+      "per_class": {
+        "Legitimate": {
+          "target_rows": 2000,
+          "eligible_rows": 4621,
+          "fraction": 0.4328,
+          "moved_rows": 1999,
+          "split": {
+            "val_fraction_target": 0.43280675178532785,
+            "val_fraction_achieved": 0.4326,
+            "clusters_total": 3532,
+            "clusters_mixed_stratum": 4,
+            "largest_cluster": 139,
+            "per_stratum": {
+              "Assassin|0": {
+                "rows": 380,
+                "clusters": 307,
+                "val_target": 164,
+                "val_rows": 164,
+                "val_fraction": 0.4316,
+                "clusters_too_large_for_val": 0
+              },
+              "CEAS-08|0": {
+                "rows": 1697,
+                "clusters": 1318,
+                "val_target": 734,
+                "val_rows": 734,
+                "val_fraction": 0.4325,
+                "clusters_too_large_for_val": 0
+              },
+              "TREC-07|0": {
+                "rows": 2544,
+                "clusters": 1907,
+                "val_target": 1101,
+                "val_rows": 1101,
+                "val_fraction": 0.4328,
+                "clusters_too_large_for_val": 0
+              }
+            }
+          }
+        },
+        "Spam": {
+          "target_rows": 700,
+          "eligible_rows": 1155,
+          "fraction": 0.6061,
+          "moved_rows": 700,
+          "split": {
+            "val_fraction_target": 0.6060606060606061,
+            "val_fraction_achieved": 0.6061,
+            "clusters_total": 956,
+            "clusters_mixed_stratum": 24,
+            "largest_cluster": 20,
+            "per_stratum": {
+              "Assassin|2": {
+                "rows": 132,
+                "clusters": 96,
+                "val_target": 80,
+                "val_rows": 80,
+                "val_fraction": 0.6061,
+                "clusters_too_large_for_val": 0
+              },
+              "Enron|2": {
+                "rows": 1023,
+                "clusters": 860,
+                "val_target": 620,
+                "val_rows": 620,
+                "val_fraction": 0.6061,
+                "clusters_too_large_for_val": 0
+              }
+            }
+          }
+        }
+      },
+      "val_rows_linked_to_holdout_info": 0
+    },
+    "counts": {
+      "finetune_train": {
+        "Assassin": {
+          "0": 3705,
+          "2": 1388
+        },
+        "CEAS-08": {
+          "-1": 18631,
+          "0": 15547
+        },
+        "Enron": {
+          "0": 12741,
+          "2": 11482
+        },
+        "Nazario": {
+          "1": 1159
+        },
+        "Nigerian_Fraud": {
+          "1": 1991
+        },
+        "TREC-05": {
+          "-1": 18723,
+          "0": 27663
+        },
+        "TREC-07": {
+          "-1": 25451,
+          "0": 21782
+        }
+      },
+      "finetune_val": {
+        "Assassin": {
+          "0": 216,
+          "2": 64
+        },
+        "CEAS-08": {
+          "-1": 2184,
+          "0": 963
+        },
+        "Enron": {
+          "0": 1381,
+          "2": 699
+        },
+        "Nazario": {
+          "1": 126
+        },
+        "Nigerian_Fraud": {
+          "1": 214
+        },
+        "TREC-05": {
+          "-1": 2052,
+          "0": 2997
+        },
+        "TREC-07": {
+          "-1": 2943,
+          "0": 1445
+        }
+      },
+      "test_holdout": {
+        "TREC-06": {
+          "-1": 3894,
+          "0": 12408
+        }
+      },
+      "test_holdout3": {
+        "Assassin": {
+          "0": 164,
+          "2": 72
+        },
+        "CEAS-08": {
+          "0": 736
+        },
+        "Enron": {
+          "2": 628
+        },
+        "Nazario": {
+          "1": 222
+        },
+        "Nigerian_Fraud": {
+          "1": 299
+        },
+        "TREC-07": {
+          "0": 1099
+        }
+      }
+    },
+    "totals": {
+      "finetune_train": {
+        "rows": 160263,
+        "by_label3": {
+          "-1": 62805,
+          "0": 81438,
+          "1": 3150,
+          "2": 12870
+        },
+        "by_label_set": {
+          "legit": 81438,
+          "phish": 3150,
+          "spam": 12870,
+          "unwanted": 62805
+        },
+        "binary_positives": 78825,
+        "binary_base_rate": 0.4918,
+        "rows_with_full_label3": 97458
+      },
+      "finetune_val": {
+        "rows": 15284,
+        "by_label3": {
+          "-1": 7179,
+          "0": 7002,
+          "1": 340,
+          "2": 763
+        },
+        "by_label_set": {
+          "legit": 7002,
+          "phish": 340,
+          "spam": 763,
+          "unwanted": 7179
+        },
+        "binary_positives": 8282,
+        "binary_base_rate": 0.5419,
+        "rows_with_full_label3": 8105
+      },
+      "test_holdout": {
+        "rows": 16302,
+        "by_label3": {
+          "-1": 3894,
+          "0": 12408
+        },
+        "by_label_set": {
+          "legit": 12408,
+          "unwanted": 3894
+        },
+        "binary_positives": 3894,
+        "binary_base_rate": 0.2389,
+        "rows_with_full_label3": 12408
+      },
+      "test_holdout3": {
+        "rows": 3220,
+        "by_label3": {
+          "0": 1999,
+          "1": 521,
+          "2": 700
+        },
+        "by_label_set": {
+          "legit": 1999,
+          "phish": 521,
+          "spam": 700
+        },
+        "binary_positives": 1221,
+        "binary_base_rate": 0.3792,
+        "rows_with_full_label3": 3220
+      }
+    },
+    "shingle_links": {
+      "holdout_test3": 0,
+      "holdout_train": 0,
+      "holdout_val": 0,
+      "test3_train": 0,
+      "test3_val": 0,
+      "train_val": 0,
+      "holdout_pretrain": 0,
+      "val_pretrain": 5336,
+      "test3_pretrain": 0
+    },
+    "near_dup": {
+      "shingle_chars": 50,
+      "stride": 25,
+      "window_chars": 1500,
+      "min_shared": 3,
+      "cap": 2000
+    },
+    "leakage": {
+      "holdout_vs_train_key_rows": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout_vs_val_key_rows": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout_vs_pretrain_key_rows": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": "pretrain keys re-derived from stored text (text_identity)"
+      },
+      "holdout_vs_train_shingle_links": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout_vs_val_shingle_links": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout_vs_pretrain_shingle_links": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": "pretrain bases re-derived from stored text (text_identity)"
+      },
+      "train_vs_val_k_full_rows": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "train_vs_val_any_key_rows": {
+        "count": 0,
+        "asserted": false,
+        "pass": true,
+        "detail": ""
+      },
+      "train_vs_val_shingle_links": {
+        "count": 0,
+        "asserted": false,
+        "pass": true,
+        "detail": ""
+      },
+      "val_vs_pretrain_key_rows": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "val_vs_pretrain_shingle_links": {
+        "count": 5336,
+        "asserted": false,
+        "pass": true,
+        "detail": "near-twins of val emails in MLM text; counted, not removed (val is model selection, the holdout is the gate)"
+      },
+      "train_vs_pretrain_key_overlap": {
+        "count": 102100,
+        "asserted": false,
+        "pass": true,
+        "detail": "allowed: TREC-05/Enron ham are Enron mail"
+      },
+      "holdout_rows_in_train_or_val_by_source": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "test_holdout_rows_from_other_sources": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "pretrain_rows": {
+        "count": 226273,
+        "asserted": false,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout3_vs_train_key_rows": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout3_vs_val_key_rows": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout3_vs_pretrain_key_rows": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": "pretrain keys re-derived from stored text (text_identity)"
+      },
+      "holdout3_vs_holdout_key_rows": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout3_vs_train_shingle_links": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout3_vs_val_shingle_links": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout3_vs_pretrain_shingle_links": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": "pretrain bases re-derived from stored text (text_identity)"
+      },
+      "holdout3_vs_holdout_shingle_links": {
+        "count": 0,
+        "asserted": false,
+        "pass": true,
+        "detail": "two test sets; shared 1998-2008 spam templates are allowed"
+      },
+      "holdout3_rows_without_full_label3": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "holdout3_rows_from_holdout_source": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "doc_id_shared_between_splits": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": ""
+      },
+      "label3_contract_violations": {
+        "count": 0,
+        "asserted": true,
+        "pass": true,
+        "detail": "{}"
+      },
+      "holdout3_rows": {
+        "count": 3220,
+        "asserted": false,
+        "pass": true,
+        "detail": ""
+      }
+    },
+    "files": {
+      "finetune_train": "/home/nathank/BITS/dev/Phishing Model/data/processed/finetune_train.parquet",
+      "finetune_val": "/home/nathank/BITS/dev/Phishing Model/data/processed/finetune_val.parquet",
+      "test_holdout": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet",
+      "test_holdout3": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout3.parquet"
+    },
+    "config_snapshot": {
+      "positive_label3_by_source": {
+        "Assassin": 2,
+        "Enron": 2,
+        "Ling": 2,
+        "TREC-05": -1,
+        "TREC-06": -1,
+        "TREC-07": -1,
+        "CEAS-08": -1,
+        "Nazario": 1,
+        "Nigerian_Fraud": 1
+      },
+      "champa": {
+        "repo": "kudzaiprichard/aura-phishing-email-corpus",
+        "files": [
+          "Nazario.csv",
+          "Nigerian_Fraud.csv"
+        ],
+        "local_dir": "data/raw/aura",
+        "label3": 1
+      },
+      "bigshp": {
+        "enabled": false,
+        "verdict": "DO_NOT_USE",
+        "repo": "locuoco/the-biggest-spam-ham-phish-email-dataset-300000",
+        "file": "df.csv",
+        "local_dir": "data/raw/bigshp",
+        "drop_phish_class": true,
+        "min_chars": 160,
+        "min_english_fraction": 0.7,
+        "near_dup_token_key_len": 30
+      },
+      "new_source_test_fraction": 0.15,
+      "new_source_val_fraction": 0.1,
+      "holdout3_from_val": {
+        "legit_sources": [
+          "CEAS-08",
+          "TREC-07",
+          "Assassin"
+        ],
+        "spam_sources": [
+          "Assassin",
+          "Enron"
+        ],
+        "legit_rows": 2000,
+        "spam_rows": 700
+      }
+    }
+  }
+}
diff --git a/presentation/data/processed/tokenizer_report.json b/presentation/data/processed/tokenizer_report.json
new file mode 100644
index 0000000000000000000000000000000000000000..95f20f006b953836208c33096fc5931e14db1c68
--- /dev/null
+++ b/presentation/data/processed/tokenizer_report.json
@@ -0,0 +1,384 @@
+{
+  "train": {
+    "tokenizer_json": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+    "tokenizer_sha1": "9f50c5e775089d2ca3efc82e100cd6b3396ed7e7",
+    "requested_vocab_size": 32768,
+    "final_vocab_size": 32768,
+    "min_frequency": 2,
+    "special_token_ids": {
+      "[PAD]": 0,
+      "[UNK]": 1,
+      "[CLS]": 2,
+      "[SEP]": 3,
+      "[MASK]": 4
+    },
+    "normalizer": "NFKC (case preserved)",
+    "pre_tokenizer": "ByteLevel(add_prefix_space=False)",
+    "training_files": {
+      "pretrain.parquet": {
+        "rows_in_file": 226273,
+        "rows_used": 226273,
+        "chars": 407902364
+      },
+      "finetune_train.parquet": {
+        "rows_in_file": 157113,
+        "rows_used": 157113,
+        "chars": 278034593
+      }
+    },
+    "limit_docs_per_file": null,
+    "excluded_files": [
+      "finetune_val.parquet",
+      "test_holdout.parquet"
+    ],
+    "holdout_exact_collisions": {
+      "pretrain.parquet": 0,
+      "finetune_train.parquet": 0
+    },
+    "train_seconds": 28.0
+  },
+  "stats": {
+    "max_seq_len": 512,
+    "vocab_size": 32768,
+    "pretrain_sample_rows": 20000,
+    "splits": [
+      {
+        "split": "pretrain",
+        "file": "pretrain.parquet",
+        "rows": 20000,
+        "mean_tokens": 476.4,
+        "median_tokens": 198.0,
+        "p95_tokens": 1560.0499999999993,
+        "max_tokens": 59376,
+        "chars_per_token": 3.76,
+        "pct_truncated": 20.52,
+        "unk_rate": 0.0,
+        "mean_tokens_by_label": {}
+      },
+      {
+        "split": "finetune_train",
+        "file": "finetune_train.parquet",
+        "rows": 157113,
+        "mean_tokens": 486.9,
+        "median_tokens": 227.0,
+        "p95_tokens": 1540.0,
+        "max_tokens": 886635,
+        "chars_per_token": 3.635,
+        "pct_truncated": 24.05,
+        "unk_rate": 0.0,
+        "mean_tokens_by_label": {
+          "0": 620.9,
+          "1": 342.7
+        }
+      },
+      {
+        "split": "finetune_val",
+        "file": "finetune_val.parquet",
+        "rows": 17643,
+        "mean_tokens": 453.7,
+        "median_tokens": 200.0,
+        "p95_tokens": 1420.8999999999978,
+        "max_tokens": 128702,
+        "chars_per_token": 3.583,
+        "pct_truncated": 22.31,
+        "unk_rate": 0.0,
+        "mean_tokens_by_label": {
+          "0": 594.4,
+          "1": 307.3
+        }
+      },
+      {
+        "split": "test_holdout",
+        "file": "test_holdout.parquet",
+        "rows": 16302,
+        "mean_tokens": 708.6,
+        "median_tokens": 244.0,
+        "p95_tokens": 1253.949999999999,
+        "max_tokens": 878138,
+        "chars_per_token": 3.432,
+        "pct_truncated": 21.93,
+        "unk_rate": 0.0,
+        "mean_tokens_by_label": {
+          "0": 842.2,
+          "1": 282.9
+        }
+      }
+    ],
+    "examples": [
+      {
+        "name": "business",
+        "text": "Re: Q3 budget review\n\nHi Sarah, can we move the budget review to Thursday at 2pm? I've attached the revised forecast. Thanks, Mike",
+        "n_tokens_with_specials": 35,
+        "tokens": [
+          "[CLS]",
+          "Re",
+          ":",
+          "\u0120Q",
+          "3",
+          "\u0120budget",
+          "\u0120review",
+          "\u010a",
+          "\u010a",
+          "Hi",
+          "\u0120Sarah",
+          ",",
+          "\u0120can",
+          "\u0120we",
+          "\u0120move",
+          "\u0120the",
+          "\u0120budget",
+          "\u0120review",
+          "\u0120to",
+          "\u0120Thursday",
+          "\u0120at",
+          "\u01202",
+          "pm",
+          "?",
+          "\u0120I",
+          "'ve",
+          "\u0120attached",
+          "\u0120the",
+          "\u0120revised",
+          "\u0120forecast",
+          ".",
+          "\u0120Thanks",
+          ",",
+          "\u0120Mike",
+          "[SEP]"
+        ],
+        "decoded": "Re: Q3 budget review\n\nHi Sarah, can we move the budget review to Thursday at 2pm? I've attached the revised forecast. Thanks, Mike",
+        "round_trip_exact": true,
+        "round_trip_matches_nfkc": true
+      },
+      {
+        "name": "url_heavy",
+        "text": "Your account has been suspended\n\nVerify now: http://secure-login.paypa1-verify.com/account/update?id=8f3a9c2e&session=XyZ123 or https://bit.ly/3xYz9Ab before 24 hours.",
+        "n_tokens_with_specials": 64,
+        "tokens": [
+          "[CLS]",
+          "Your",
+          "\u0120account",
+          "\u0120has",
+          "\u0120been",
+          "\u0120suspended",
+          "\u010a",
+          "\u010a",
+          "Ver",
+          "ify",
+          "\u0120now",
+          ":",
+          "\u0120http",
+          "://",
+          "secure",
+          "-",
+          "login",
+          ".",
+          "pay",
+          "pa",
+          "1",
+          "-",
+          "ver",
+          "ify",
+          ".",
+          "com",
+          "/",
+          "account",
+          "/",
+          "update",
+          "?",
+          "id",
+          "=",
+          "8",
+          "f",
+          "3",
+          "a",
+          "9",
+          "c",
+          "2",
+          "e",
+          "&",
+          "session",
+          "=",
+          "X",
+          "yZ",
+          "123",
+          "\u0120or",
+          "\u0120https",
+          "://",
+          "bit",
+          ".",
+          "ly",
+          "/",
+          "3",
+          "x",
+          "Yz",
+          "9",
+          "Ab",
+          "\u0120before",
+          "\u012024",
+          "\u0120hours",
+          ".",
+          "[SEP]"
+        ],
+        "decoded": "Your account has been suspended\n\nVerify now: http://secure-login.paypa1-verify.com/account/update?id=8f3a9c2e&session=XyZ123 or https://bit.ly/3xYz9Ab before 24 hours.",
+        "round_trip_exact": true,
+        "round_trip_matches_nfkc": true
+      },
+      {
+        "name": "all_caps",
+        "text": "URGENT!!! FINAL NOTICE\n\nYOUR MAILBOX IS FULL. CLICK HERE TO UPGRADE YOUR STORAGE IMMEDIATELY OR LOSE ALL MESSAGES.",
+        "n_tokens_with_specials": 33,
+        "tokens": [
+          "[CLS]",
+          "URGENT",
+          "!!!",
+          "\u0120FINAL",
+          "\u0120NOTICE",
+          "\u010a",
+          "\u010a",
+          "YOUR",
+          "\u0120MAIL",
+          "B",
+          "OX",
+          "\u0120IS",
+          "\u0120FULL",
+          ".",
+          "\u0120CLICK",
+          "\u0120HERE",
+          "\u0120TO",
+          "\u0120U",
+          "PG",
+          "R",
+          "ADE",
+          "\u0120YOUR",
+          "\u0120STOR",
+          "AGE",
+          "\u0120IMMEDI",
+          "ATELY",
+          "\u0120OR",
+          "\u0120LO",
+          "SE",
+          "\u0120ALL",
+          "\u0120MESSAGES",
+          ".",
+          "[SEP]"
+        ],
+        "decoded": "URGENT!!! FINAL NOTICE\n\nYOUR MAILBOX IS FULL. CLICK HERE TO UPGRADE YOUR STORAGE IMMEDIATELY OR LOSE ALL MESSAGES.",
+        "round_trip_exact": true,
+        "round_trip_matches_nfkc": true
+      },
+      {
+        "name": "unicode_lookalike",
+        "text": "\uff2d\uff49\uff43\uff52\uff4f\uff53\uff4f\uff46\uff54 365 \u2014 Password expiry \u26a0\ufe0f\n\nDear user, your pass\u00adword expires today. Caf\u00e9, na\u00efve, \u6771\u4eac office.",
+        "n_tokens_with_specials": 43,
+        "tokens": [
+          "[CLS]",
+          "Microsoft",
+          "\u0120365",
+          "\u0120\u00e2\u0122\u0136",
+          "\u0120Password",
+          "\u0120exp",
+          "iry",
+          "\u0120\u00e2",
+          "\u013c",
+          "\u0142",
+          "\u00ef",
+          "\u00b8",
+          "\u0131",
+          "\u010a",
+          "\u010a",
+          "Dear",
+          "\u0120user",
+          ",",
+          "\u0120your",
+          "\u0120pass",
+          "\u00c2",
+          "\u0143",
+          "word",
+          "\u0120expires",
+          "\u0120today",
+          ".",
+          "\u0120C",
+          "af",
+          "\u00c3\u00a9",
+          ",",
+          "\u0120na",
+          "\u00c3",
+          "\u00af",
+          "ve",
+          ",",
+          "\u0120",
+          "\u00e6\u013f",
+          "\u00b1",
+          "\u00e4\u00ba",
+          "\u00ac",
+          "\u0120office",
+          ".",
+          "[SEP]"
+        ],
+        "decoded": "Microsoft 365 \u2014 Password expiry \u26a0\ufe0f\n\nDear user, your pass\u00adword expires today. Caf\u00e9, na\u00efve, \u6771\u4eac office.",
+        "round_trip_exact": false,
+        "round_trip_matches_nfkc": true
+      },
+      {
+        "name": "html_money",
+        "text": "<html><body><p>Dear Friend,</p><p>I am Barrister James. Transfer of US$15,500,000.00 &amp; 30% for you.</p></body></html>",
+        "n_tokens_with_specials": 49,
+        "tokens": [
+          "[CLS]",
+          "<",
+          "html",
+          "><",
+          "body",
+          "><",
+          "p",
+          ">",
+          "Dear",
+          "\u0120Friend",
+          ",",
+          "</",
+          "p",
+          "><",
+          "p",
+          ">",
+          "I",
+          "\u0120am",
+          "\u0120Bar",
+          "ris",
+          "ter",
+          "\u0120James",
+          ".",
+          "\u0120Transfer",
+          "\u0120of",
+          "\u0120US",
+          "$",
+          "15",
+          ",",
+          "500",
+          ",",
+          "000",
+          ".",
+          "00",
+          "\u0120&",
+          "amp",
+          ";",
+          "\u012030",
+          "%",
+          "\u0120for",
+          "\u0120you",
+          ".</",
+          "p",
+          "></",
+          "body",
+          "></",
+          "html",
+          ">",
+          "[SEP]"
+        ],
+        "decoded": "<html><body><p>Dear Friend,</p><p>I am Barrister James. Transfer of US$15,500,000.00 &amp; 30% for you.</p></body></html>",
+        "round_trip_exact": true,
+        "round_trip_matches_nfkc": true
+      }
+    ]
+  }
+}
diff --git a/presentation/docs/research/research_model-math.md b/presentation/docs/research/research_model-math.md
new file mode 100644
index 0000000000000000000000000000000000000000..ce9ae1ba8713f8c2db7505ddb3128820506a183b
--- /dev/null
+++ b/presentation/docs/research/research_model-math.md
@@ -0,0 +1,323 @@
+# model-math
+
+## SUMMARY
+Config A (24 layers, d=1024, 16 heads, FFN 4096, V=32768, Lmax=512, pre-LN, tied MLM head) is the recommended pin: 337,476,608 params for the pretraining model (302.3M encoder + 34.1M embeddings + 1.08M MLM head), 337,443,842 for the deployed classifier; it is +2.3% over the 330M target, and the nearest-to-330M alternatives are 23 layers (324.9M) or V=16384 (320.7M). Training state is 16 B/param = 5.03 GiB (+0.63 GiB autocast bf16 weight cache); with SDPA flash/mem-efficient attention the per-token-per-layer activation footprint is about 38·d bytes under torch.autocast (34·d if the residual stream is bf16), versus Korthikanti's 34·d + 5·a·s for eager attention, so on the 24 GiB card with 1.9 GiB of desktop use and a 4 GiB headroom target the no-checkpoint micro-batch is 16 (24 is marginal), while full per-layer checkpointing fits 48+ with 4 GiB to spare; computing MLM logits only on the ~15% masked positions saves 85% of the logits/cross-entropy memory (136 MiB per sample at B=1). The RTX 3090 Ti bf16 tensor-core peak is 80 TFLOPS dense with FP32 accumulate (160 sparse; 40 TFLOPS is the non-tensor rate) — derived from the GA102 whitepaper's RTX 3090 figure of 71/142 scaled by SM count and clock, and consistent with Wikipedia's 320 sparse FP16-accumulate figure halved for FP32 accumulate; at 2.03 GFLOP/token (6N + 14·nl·d·L + masked head) and 30-45% MFU that is 11.8k-17.8k tokens/s, i.e. 2.3-3.5 h per epoch of 150M tokens (1,144 optimizer steps at B_eff=256) and 6.3-9.4 h for 400M, with full checkpointing costing x1.33. Batch-1 inference of a 512-token email is ~336 GFLOP: ~14 ms on the 3090 Ti at 30% MFU but ~336 ms on the 5950X at 1 TFLOPS effective, so the 74 ms/sample benchmark is only met on a GPU (ZeroGPU runs on RTX Pro 6000 Blackwell slices) or with truncation to ~128 tokens on CPU; real emails averaging 200-300 tokens make dynamic padding worth roughly 2x. Recommended training recipe: checkpointing off by default, micro-batch 16 x 16 accumulation = 256, AdamW peak LR 1.5e-4 (range 1e-4 to 2e-4), warmup min(1000 steps, 6% of total), betas (0.9, 0.98), eps 1e-6, weight decay 0.01, clip 1.0, 15% dynamic masking 80/10/10, sequence packing for pretraining (also keeps the Flash SDPA backend eligible because it needs no attn_mask) and length-bucketed dynamic padding for fine-tuning and inference.
+
+## FINDINGS
+# Architecture / parameter / VRAM / throughput math for the from-scratch encoder
+
+Calculator: `/tmp/claude-1000/-home-nathank-BITS-dev-Phishing-Model/8697d948-f234-4e8c-a1e6-c0b6f6ddd7ef/scratchpad/model_math.py` (pure Python 3.14.7, no torch; full output in `model_math.out` next to it and pasted in section 7). Nothing was installed and nothing under the project directory was touched.
+
+## 0. Hardware facts verified live (needed before any throughput number)
+
+- **RTX 3090 Ti**: 84 SMs, 10,752 CUDA cores, 336 tensor cores, boost 1860 MHz, FP32 39.99 TFLOPS, "Tensor compute (FP16)" 320 TFLOPS *sparse* (table formula: tensor cores x clock x 256/1000 = FP16-accumulate), 24 GB @ 1008 GB/s, 450 W — https://en.wikipedia.org/wiki/GeForce_RTX_30_series
+- **GA102 whitepaper Table 9 (RTX 3090 FE, 82 SMs @ 1695 MHz)**: "Peak FP16 Tensor TFLOPS with FP16 Accumulate 142/284", "Peak FP16 Tensor TFLOPS with FP32 Accumulate 71/142", "Peak BF16 Tensor TFLOPS with FP32 Accumulate 71/142" (second number = sparsity). The RTX A6000 rows (same GA102, 84 SMs) show 154.8/309.6 for *both* FP16- and FP32-accumulate, i.e. **GeForce GA10x runs FP32-accumulate tensor math at half rate; pro cards do not**. No prose explains it; the table rows are the evidence — https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-architecture-whitepaper-v2.pdf (Appendix A, p.44)
+- **Derived 3090 Ti bf16 dense peak**: 71 x (84x1860)/(82x1695) = **79.8 ≈ 80 TFLOPS** (160 sparse). Cross-check: 336 x 1.860 x 256 = 160 TFLOPS FP16-accumulate dense, halved for FP32 accumulate = 80. **Answer to "80 or 40?": 80 TFLOPS dense bf16 (FP32 acc); 40 TFLOPS is the non-tensor FP32/BF16 rate.** There is no bf16-with-bf16-accumulate mode in the table, so bf16 GEMMs are always the 80 figure.
+- PyTorch bf16 GEMMs: `torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction` "is set to True by default for BF16" (affects split-K reductions only; tensor-core accumulation is fp32) — https://docs.pytorch.org/docs/2.14/notes/cuda.html. Same page: `memory_allocated` vs `memory_reserved`, and `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` to reduce fragmentation when batch shapes change.
+- TechPowerUp returned HTTP 403 (not used). Everything attributed to Context7 was cross-checked against docs.pytorch.org 2.14 pages (upstream wins).
+
+## 1. Exact parameter counts
+
+Formulas (stated assumptions): token emb V·d; position emb Lmax·d; emb LN 2d; per layer QKV 3d²+3d, out-proj d²+d, 2 LN 4d, FFN up d·f+f, FFN down f·d+d; final LN 2d (pre-LN needs it); MLM head dense d²+d + LN 2d + output bias V (tied) [untied: +V·d]; classifier d²+d + 2d+2; no segment embeddings.
+
+Sanity check: applying the same per-layer/embedding formulas to BERT-base (post-LN, V=30522, +2 segment rows, +pooler) gives exactly **109,482,240**, which matches `BertModel('bert-base-uncased').num_parameters()` (HF figure from memory, UNVERIFIED live; the exact match is strong evidence the formulas are right). BERT paper quotes 110M/340M — https://ar5iv.labs.arxiv.org/html/1810.04805
+
+| config | per-layer | encoder (+final LN) | embeddings | MLM head tied | **TOTAL tied** | untied | fine-tune (cls head) | N_nonemb |
+|---|---|---|---|---|---|---|---|---|
+| **A 24L d1024 h16 f4096 V32768** | 12,596,224 | 302,311,424 | 34,080,768 | 1,084,416 | **337,476,608** | 371.0M | **337,443,842** | 303.4M |
+| B 24L V32000 | 12,596,224 | 302,311,424 | 33,294,336 | 1,083,648 | 336,689,408 | 369.5M | 336.7M | 303.4M |
+| C 24L V16384 | 12,596,224 | 302,311,424 | 17,303,552 | 1,068,032 | 320,683,008 | 337.5M | 320.7M | 303.4M |
+| D 20L V32768 | 12,596,224 | 251,926,528 | 34,080,768 | 1,084,416 | 287,091,712 | 320.6M | 287.1M | 253.0M |
+| E 22L V32768 | 12,596,224 | 277,118,976 | 34,080,768 | 1,084,416 | 312,284,160 | 345.8M | 312.3M | 278.2M |
+| F 24L d768 h12 f3072 V32768 | 7,087,872 | 170,110,464 | 25,560,576 | 624,896 | 196,295,936 | 221.5M | 196.3M | 170.7M |
+| A' 23L d1024 V32768 (extra) | 12,596,224 | 289,715,200 | 34,080,768 | 1,084,416 | 324,880,384 | 358.4M | 324.8M | 290.8M |
+
+Config A breakdown: layer = qkv 3,148,800 + out_proj 1,049,600 + 2xLN 4,096 + ffn_up 4,198,400 + ffn_down 4,195,328; embeddings = tok 33,554,432 + pos 524,288 + LN 2,048; MLM head tied = dense 1,049,600 + LN 2,048 + bias 32,768; classifier head = 1,051,650.
+
+Distance from 330M (tied total): A' 23L −5.12M; B +6.69M; **A +7.48M**; C −9.32M; E −17.72M; D −42.91M; F −133.70M. **Recommend A** (BERT-large shape, 2^15 vocab, +2.3% over target; BERT-large itself is 335–340M). If the brief enforces ≤330M strictly, use A' (23L, 324.9M) rather than C (16K vocab fragments URLs/headers badly). Deployed classifier = 337,443,842 params → 675 MB bf16 / 1.35 GB fp32 safetensors.
+
+Vocab alignment: 32768 = 2^15 (multiple of 64 and 128), 32000 = 64·500 = 128·250, 16384 = 2^14 — all fine. NVIDIA guidance: for FP16/BF16 "matrix dimensions would need to be multiples of 8 elements for best efficiency (or 64 elements on A100)", and "choosing multiples of larger powers of two up to 128 bytes (64 for FP16) can further improve efficiency"; tile and wave quantization explain the cliffs — https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html
+
+## 2. Memory
+
+### Static (AdamW, bf16 autocast, fp32 master params)
+16 B/param (fp32 params 4 + fp32 grads 4 + Adam m,v 8) x 337,476,608 = **5.03 GiB (5.40 GB)**; plus autocast's weight-cast cache (`cache_enabled`, bf16 copies of parameters inside the autocast region) ≈ 2 B/param = 0.63 GiB → **5.66 GiB**. Add CUDA context 0.5–1 GiB and the 1.9 GiB already used by kwin/plasma. GPU = 24,564 MiB = 23.99 GiB. Budget left for activations+logits with a 4 GiB headroom target: **11.68 GiB** (process cap 18.09 GiB).
+
+### Per-token-per-layer activation bytes saved for backward (pre-LN, autocast bf16, SDPA)
+Korthikanti et al. 2022 Eq.1: "Activations memory per layer = sbh(34 + 5as/h)" for fp16 eager attention; itemized: attention 11sbh + 5as²b (QKV input 2sbh, Q,K 4sbh, softmax out 2as²b, dropout mask as²b, dropout out 2as²b, V 2sbh, attn-over-V out 2sbh, out dropout mask sbh), MLP 19sbh (input 2, GELU input 8, GELU output 8, mask 1), LNs 4sbh — https://ar5iv.labs.arxiv.org/html/2205.05198. Per token per layer that is 34d + 5·a·s = 34,816 + 40,960 = 75,776 B at d=1024, a=16, s=512 — the L×L terms would be **+118% on top of 34d**.
+
+FlashAttention stores only "the output O and the softmax normalization statistics (m,ℓ)" and recomputes S and P in backward; dropout keeps "the pseudo-random number generator state" instead of a mask; memory is linear in sequence length — https://ar5iv.labs.arxiv.org/html/2205.14135. PyTorch SDPA dispatches to FlashAttention-2 / Memory-Efficient / C++ math — https://docs.pytorch.org/docs/2.14/generated/torch.nn.functional.scaled_dot_product_attention.html. FlashAttention-2 supports "Ampere, Ada, or Hopper GPUs (e.g., A100, RTX 3090, RTX 4090, H100)", "fp16 and bf16", "all head dimensions up to 256" — https://github.com/Dao-AILab/flash-attention (3090 Ti is sm86 Ampere, head_dim 64: eligible).
+
+My tally (config A), tensors actually saved:
+
+| tensor | fp32 residual (autocast) | bf16 residual |
+|---|---|---|
+| LN1 input = residual stream | 4d | 2d |
+| QKV linear input (bf16 LN1 out) | 2d | 2d |
+| Q,K,V bf16 (saved by SDPA) | 6d | 6d |
+| attn output O bf16 (SDPA bwd + out-proj input, one tensor) | 2d | 2d |
+| SDPA logsumexp fp32 | 4h (64 B) | 4h |
+| out-proj dropout mask (bool) | d | d |
+| LN2 input = residual stream | 4d | 2d |
+| FFN-up input (bf16 LN2 out) | 2d | 2d |
+| FFN-up out = GELU in (bf16, width 4d) | 8d | 8d |
+| GELU out = FFN-down in (bf16) | 8d | 8d |
+| FFN-down dropout mask | d | d |
+| **total** | **38,976 B = 38.06 d (c = 38 + 4h/d)** | **34,880 B = 34.06 d (c = 34)** |
+
+Why 38 not 34: under `torch.autocast` `layer_norm`, `softmax`, `log_softmax`, `cross_entropy`, `nll_loss` are on the "autocast to float32" list while `linear`/`matmul` are on the float16/bf16 list — https://docs.pytorch.org/docs/2.14/amp.html. Embedding lookups and residual adds are not autocast ops, so the residual stream stays fp32 and each LN saves a 4-byte input (+2d per LN). Use **c = 38** for planning; c = 34 only if the model is run with a bf16 residual stream. An eager softmax under autocast runs in fp32, so an accidental math-backend fallback costs ≥7·a·s per token per layer (worse than the paper's 5·a·s).
+
+Non-layer per token: 9d (fp32 emb output/emb-LN input 4d, emb dropout mask d, final-LN input 4d). MLM head per *scored* position: 12d + logits 2V (bf16) + fp32 cross-entropy intermediates (4V saved log_softmax output + 4V transient fp32 up-cast) = **10V B/position at peak**. Per 512-token sample: no-ckpt layers 457 MiB (c=38) / 409 MiB (c=34); full-ckpt fp32 layer inputs 48 MiB (+19 MiB one live layer) / bf16 inputs 24 MiB; logits full 160 MiB vs masked ~15% (76 pos) 24 MiB → **gather-before-head saves 136 MiB/sample (85%)**.
+
+### Fit tables (config A, L=512; totals include desktop 1.9 + ctx 0.75 + static 5.66 + 10% allocator slack on activations; FITS = ≥4 GiB headroom, tight = 2–4, NO < 2)
+
+(i) no checkpointing, c=38d (autocast): B=4 masked 10.39 GiB total (13.60 headroom) FITS; B=8 12.47 (11.52) FITS; **B=16 masked 16.63 (7.36) FITS**, B=16 full-logits 18.84 (5.15) FITS; **B=24 masked 20.78 (3.20) tight**, B=24 full 24.10 NO; B=32 masked 24.94 NO; B=48 NO.
+(i') c=34d (bf16 residual): B=16 masked 15.80 (8.19) FITS; B=24 masked 19.55 (4.44) FITS; B=32 masked 23.29 (0.69) NO.
+(ii) full per-layer checkpointing, fp32 layer inputs 4d/tok/layer + one live layer: B=16 masked 9.93 (14.06); B=32 masked 11.55 (12.44); **B=48 masked 13.17 (10.82) FITS**, B=48 full-logits 19.81 (4.18) FITS.
+(ii') checkpointing with bf16 layer inputs (2d, the task's assumption): B=48 masked 11.83 (12.16) FITS; B=48 full 18.46 (5.52) FITS.
+
+**Conclusion**: within a 14–18 GiB process budget and ≥4 GiB headroom, (i) no-checkpoint → **micro-batch 16** (24 only if the residual is bf16 or the desktop frees memory); (ii) full checkpointing → **48 fits with >10 GiB spare, 64–96 would too**. Checkpointing is not needed for memory at B=16 and costs x1.33 compute, so default it OFF.
+
+## 3. Training throughput
+
+FLOPs/token: Kaplan: N "excluding all vocabulary and positional embeddings", "C_forward ≈ 2N + 2 n_layer n_ctx d_model", "C ≈ 6NBS" — https://ar5iv.labs.arxiv.org/html/2001.08361. Chinchilla: "FLOPs(N,D) ≈ 6ND", Appendix F exact count within 0.99–1.10 of 6ND — https://ar5iv.labs.arxiv.org/html/2203.15556. PaLM App. B: "6N matmul FLOPs per token seen (2N for forward and 4N for backward)" and "dense self-attention add 6LH(2QT) FLOPs per token", MFU = tokens/s ÷ P/(6N + 12LHQT) — https://arxiv.org/pdf/2204.02311. With flash recompute of QK^T in backward the attention term is 14·nl·d·L (12 standard).
+
+Config A, L=512, N_nonemb = 303.4M: 6N = 1.820 GF; attention 14·24·1024·512 = 0.176 GF (12·nl·d·L = 0.151 GF; 8.7% of total); tied decoder on 15% masked positions 6·V·d·0.15 = 0.030 GF (0.201 GF if scored on all 512) → **2.027 GFLOP/token** (2.198 full logits). Full checkpointing: 2.694 GFLOP/token = **x1.329**.
+
+Peak 80 TFLOPS; MFU 30 / 37.5 / 45% → 24 / 30 / 36 TFLOPS → **11,843 / 14,804 / 17,764 tokens/s**. B_eff = 256 x 512 = 131,072 tokens/step.
+
+| corpus | steps/epoch | h/epoch @30% | @37.5% | @45% | ckpt @30% | ckpt @45% |
+|---|---|---|---|---|---|---|
+| 100M | 763 | 2.35 | 1.88 | 1.56 | 3.12 | 2.08 |
+| 150M | 1,144 | 3.52 | 2.81 | 2.35 | 4.68 | 3.12 |
+| 250M | 1,907 | 5.86 | 4.69 | 3.91 | 7.79 | 5.20 |
+| 400M | 3,052 | 9.38 | 7.51 | 6.25 | 12.47 | 8.31 |
+
+A 1–2k-step warmup exceeds one epoch for corpora ≤250M at B_eff=256 → plan several epochs (BERT: 1M steps of 256x512 ≈ 40 epochs over 3.3B words — https://ar5iv.labs.arxiv.org/html/1810.04805) or use B_eff=128 to double the step count. MFU 30–45% is a planning assumption for eager PyTorch + SDPA without torch.compile; measure with `torch.cuda.Event` timing on the first 50 steps.
+
+## 4. Inference latency (fine-tuned classifier, batch 1)
+FLOPs = 2·(encoder+cls)·L + 4·nl·d·L²: L=512 → 336.4 GFLOP; 256 → 161.8; 128 → 79.3.
+- 3090 Ti bf16 @30% MFU (24 TFLOPS): **14.0 / 6.7 / 3.3 ms** (plus a kernel-launch floor of a few ms for 24 layers at batch 1 — UNVERIFIED, measure).
+- 5950X fp32 @1.0 TFLOPS effective: **336 / 162 / 79 ms**; @0.5 TFLOPS: 673 / 324 / 159 ms. (Theoretical 16 cores x 32 FLOP/cycle x ~3.9 GHz ≈ 2.0 TFLOPS; 1 TFLOPS effective ≈ 50%, UNVERIFIED assumption from the brief.)
+- vs instructor benchmark **74 ms/sample**: GPU beats it ~5x at 512 tokens; CPU misses it by ~4.5x at 512 and ~2x at 256, only borderline at 128 tokens. Real emails averaging 200–300 tokens (task-given, UNVERIFIED) halve dense cost and quarter attention cost → dynamic truncation/padding to the longest in batch (rounded to a multiple of 8) matters.
+- ZeroGPU: default `large` = "Half NVIDIA RTX Pro 6000 Blackwell, 48GB" (xlarge = full, 96GB, 2x quota); PRO can host 10 ZeroGPU Spaces, 40 min/day quota, `@spaces.GPU` default 60 s duration, Gradio SDK only, PyTorch 2.8.0–2.13.0 and Python 3.12.12 / 3.10.13 supported, `torch.compile` not supported (AOT compile only), models must be moved to `cuda` at module level — https://huggingface.co/docs/hub/spaces-zerogpu. Batch-1 latency there will be dominated by queue/cold start, not FLOPs.
+
+## 5. Recommendations to pin
+- **Model**: n_layers=24, d=1024, heads=16 (head_dim 64), ffn=4096, vocab=32768, max_len=512, pre-LN, GELU, dropout 0.1, tied MLM output, no segment embeddings → 337,476,608 params (pretrain) / 337,443,842 (classifier).
+- **Gradient checkpointing**: default OFF (B=16 fits with 7.4 GiB headroom; checkpointing costs x1.33). Expose a flag; with it ON start the batch finder at 48–64. Use `torch.utils.checkpoint.checkpoint(..., use_reentrant=False)` ("Recommended to use False"; early_stop default) — https://docs.pytorch.org/docs/2.13/checkpoint.html.
+- **Micro-batch / auto batch finder**: start at 16 (x16 accumulation = 256), probe 24 (expect tight), fall back 12 → 8. Verify with `torch.cuda.max_memory_reserved()` after 3 steps; set `PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`.
+- **Effective batch**: 256 sequences x 512 = 131,072 tokens/step (BERT's exact setting).
+- **Optimizer / LR**: AdamW, betas (0.9, 0.98), eps 1e-6 — RoBERTa: "setting β₂=0.98 to improve stability when training with large batch sizes", ε tuned, Table 9 LARGE peak LR 4e-4 at batch 8K, 30K warmup, wd 0.01, dropout 0.1 — https://ar5iv.labs.arxiv.org/html/1907.11692; BERT: LR 1e-4 at batch 256, warmup 10k, linear decay, wd 0.01 — https://ar5iv.labs.arxiv.org/html/1810.04805. Pre-LN tolerates larger LR and "the learning-rate warm-up stage can be safely removed"; their pre-LN BERT-base used 3e-4 without warmup — https://ar5iv.labs.arxiv.org/html/2002.04745. For a 24-layer pre-LN model at batch 256 from scratch: **peak 1.5e-4 (range 1e-4–2e-4), warmup min(1000 steps, 6% of total), linear or cosine decay to 10% of peak, wd 0.01 excluding biases/LN, grad-clip 1.0**. Fine-tune: LR 2e-5–5e-5, batch 16–32, 2–4 epochs (BERT ranges).
+- **MLM**: 15% dynamic masking per batch (RoBERTa), 80% [MASK] / 10% random / 10% unchanged (BERT), loss only on masked positions, gather hidden states at masked positions before the MLM head (85% head memory saving, 0.17 GFLOP/token saving).
+- **Sequence packing**: YES for pretraining — RoBERTa FULL-SENTENCES (contiguous text crossing document boundaries, no NSP; DOC-SENTENCES was slightly better but FULL-SENTENCES chosen for simplicity). Packing also removes padding and the attn_mask, which keeps the Flash backend eligible: PyTorch's dispatcher warns "Flash Attention does not support non-null attn_mask." — https://raw.githubusercontent.com/pytorch/pytorch/main/aten/src/ATen/native/transformers/sdp_utils_cpp.h; with a boolean key-padding mask SDPA uses the memory-efficient backend (still no L×L storage). NO packing for fine-tuning/inference: length-sorted buckets + dynamic padding to a multiple of 8, boolean mask broadcastable to (B,1,1,L). Flash/mem-efficient backward is non-deterministic by default (PyTorch randomness notes via Context7) — set seeds and expect small run-to-run variance.
+- **Version facts incidental to this work**: torch 2.14.1 is current on PyPI (requires_python >=3.10; cp314 manylinux_2_28 x86_64 wheel exists; depends on cuda-toolkit==13.0.3 wheels — driver 615.71 reports CUDA UMD 13.4, so compatible) — https://pypi.org/pypi/torch/2.14.1/json. torch 2.13.0 is the newest release in the ZeroGPU-supported list (cuda-toolkit 13.0.3, triton 3.7.1, cp310–cp314 wheels) — https://pypi.org/pypi/torch/2.13.0/json.
+
+## 6. Context7 vs upstream
+Context7 (/websites/pytorch_2_13) confirmed SDPBackend members (MATH, FLASH_ATTENTION, EFFICIENT_ATTENTION, CUDNN_ATTENTION), the 3-implementation note, `checkpoint(use_reentrant=False)` recommendation, and the autocast "promote"/unlisted-op rules, but did not return the autocast fp32 op list or SDPA constraints; those came from docs.pytorch.org 2.14 pages and the sdp_utils_cpp.h source as cited.
+
+## 7. Calculator output (verbatim, `python3 model_math.py`)
+```
+Python 3.14.7
+
+====================================================================================================
+1) PARAMETER COUNTS  (exact integers; M = 1e6)
+====================================================================================================
+sanity: BERT-base (post-LN, V=30522, +2 seg, +pooler) -> 109,482,240  (HF BertModel bert-base-uncased = 109,482,240)
+config                                             per-layer       encoder   embeddings  MLM head(tied)    TOTAL tied      untied    finetune    N_nonemb
+---------------------------------------------------------------------------------------------------------------------------------------------------------
+A  24L d1024 h16 f4096 V32768                     12,596,224   302,311,424   34,080,768       1,084,416   337,476,608      371.0M      337.4M      303.4M
+B  24L d1024 h16 f4096 V32000                     12,596,224   302,311,424   33,294,336       1,083,648   336,689,408      369.5M      336.7M      303.4M
+C  24L d1024 h16 f4096 V16384                     12,596,224   302,311,424   17,303,552       1,068,032   320,683,008      337.5M      320.7M      303.4M
+D  20L d1024 h16 f4096 V32768                     12,596,224   251,926,528   34,080,768       1,084,416   287,091,712      320.6M      287.1M      253.0M
+E  22L d1024 h16 f4096 V32768                     12,596,224   277,118,976   34,080,768       1,084,416   312,284,160      345.8M      312.3M      278.2M
+F  24L d768  h12 f3072 V32768                      7,087,872   170,110,464   25,560,576         624,896   196,295,936      221.5M      196.3M      170.7M
+A' 23L d1024 h16 f4096 V32768 (extra option)      12,596,224   289,715,200   34,080,768       1,084,416   324,880,384      358.4M      324.8M      290.8M
+
+config A per-layer breakdown: {'qkv': '3,148,800', 'out_proj': '1,049,600', '2xLN': '4,096', 'ffn_up': '4,198,400', 'ffn_down': '4,195,328', 'layer_total': '12,596,224'}
+config A: embeddings = tok 33,554,432 + pos 524,288 + LN 2,048; MLM head tied = dense 1,049,600 + LN 2,048 + bias 32,768 = 1,084,416; classifier head = 1,051,650
+
+distance from 330M (total, tied MLM head):
+   A' 23L d1024 h16 f4096 V32768 (extra option)       324.88M   (-5.12M)
+   B  24L d1024 h16 f4096 V32000                      336.69M   (+6.69M)
+   A  24L d1024 h16 f4096 V32768                      337.48M   (+7.48M)
+   C  24L d1024 h16 f4096 V16384                      320.68M   (-9.32M)
+   E  22L d1024 h16 f4096 V32768                      312.28M   (-17.72M)
+   D  20L d1024 h16 f4096 V32768                      287.09M   (-42.91M)
+   F  24L d768  h12 f3072 V32768                      196.30M   (-133.70M)
+vocab alignment: 32768 = 2^15 (mult of 64 and 128); 32000 = 64*500 = 128*250; 16384 = 2^14 -> all tensor-core aligned
+
+====================================================================================================
+2) ACTIVATION MEMORY MODEL  (config A, L=512, bf16 autocast + fp32 master weights, SDPA)
+====================================================================================================
+
+residual stream kept in fp32 (what torch.autocast gives you: embeddings & residual adds stay fp32)
+   LN1 input = residual stream x                                            4,096 B  =  4.00 d
+   QKV linear input (bf16 LN1 output)                                       2,048 B  =  2.00 d
+   Q,K,V bf16 (saved by SDPA for backward)                                  6,144 B  =  6.00 d
+   attn output O bf16 (SDPA bwd AND out-proj input; one tensor)             2,048 B  =  2.00 d
+   SDPA logsumexp fp32 (one per head per query)                                64 B  =  0.06 d
+   out-proj dropout mask (bool, 1 B)                                        1,024 B  =  1.00 d
+   LN2 input = residual stream x'                                           4,096 B  =  4.00 d
+   FFN-up linear input (bf16 LN2 output)                                    2,048 B  =  2.00 d
+   FFN-up output = GELU input bf16 (width f)                                8,192 B  =  8.00 d
+   GELU output = FFN-down input bf16 (width f)                              8,192 B  =  8.00 d
+   FFN-down dropout mask (bool, 1 B)                                        1,024 B  =  1.00 d
+   TOTAL per token per layer                                               38,976 B  = 38.06 d   (c = 38 + 4h/d)
+
+residual stream kept in bf16 (model.to(bf16) / Korthikanti fp16 assumption)
+   LN1 input = residual stream x                                            2,048 B  =  2.00 d
+   QKV linear input (bf16 LN1 output)                                       2,048 B  =  2.00 d
+   Q,K,V bf16 (saved by SDPA for backward)                                  6,144 B  =  6.00 d
+   attn output O bf16 (SDPA bwd AND out-proj input; one tensor)             2,048 B  =  2.00 d
+   SDPA logsumexp fp32 (one per head per query)                                64 B  =  0.06 d
+   out-proj dropout mask (bool, 1 B)                                        1,024 B  =  1.00 d
+   LN2 input = residual stream x'                                           2,048 B  =  2.00 d
+   FFN-up linear input (bf16 LN2 output)                                    2,048 B  =  2.00 d
+   FFN-up output = GELU input bf16 (width f)                                8,192 B  =  8.00 d
+   GELU output = FFN-down input bf16 (width f)                              8,192 B  =  8.00 d
+   FFN-down dropout mask (bool, 1 B)                                        1,024 B  =  1.00 d
+   TOTAL per token per layer                                               34,880 B  = 34.06 d   (c = 34 + 4h/d)
+
+Korthikanti 2022 eager-attention reference: sbh(34 + 5*a*s/h) -> per token per layer = 34d + 5*a*s = 34,816 + 40,960 = 75,776 B  -> the LxL score/softmax/dropout terms would be 118% ON TOP of 34d at L=512; SDPA flash/mem-efficient removes them (keeps only 4h B fp32 logsumexp).
+under autocast an eager softmax runs in fp32, so the eager penalty would be >= 7*a*s, i.e. even worse than the fp16 paper figure
+
+====================================================================================================
+3) VRAM BUDGET  (config A; GiB unless noted)
+====================================================================================================
+GPU: 23.99 GiB (25.76 GB)   desktop already used: 1.90   CUDA ctx: 0.75 (0.5-1)   headroom target: 4
+static training state @16 B/param x 337,476,608 = 5.03 GiB (5.40 GB); + autocast bf16 weight cache 2 B/param = 0.63 GiB -> 5.66 GiB
+=> left for activations+logits (incl. ~10% allocator slack): 11.68 GiB   (training-process cap = 18.09 GiB)
+
+per-sample (512 tok) layer activations: no-ckpt 457 MiB (c=38d) | 409 MiB (c=34d) ; full-ckpt fp32 inputs 48 MiB (+1 live layer 19.0 MiB) | bf16 inputs 24 MiB
+per-sample logits+CE at peak (10 V B/pos): full 512 pos = 160 MiB ; masked ~15% (76 pos) = 24.0 MiB  -> gather-before-head saves 136 MiB/sample (85%)
+
+(i)  NO checkpointing, c=38d (fp32 residual, autocast)
+   B     logits |  layers  other  logits  acts+10%   TOTAL  headroom  verdict
+   4  masked15% |    1.78   0.02    0.09      2.08   10.39     13.60  FITS
+   4    full512 |    1.78   0.04    0.62      2.63   10.94     13.05  FITS
+   8  masked15% |    3.57   0.04    0.19      4.16   12.47     11.52  FITS
+   8    full512 |    3.57   0.08    1.25      5.27   13.57     10.42  FITS
+  16  masked15% |    7.14   0.08    0.38      8.32   16.63      7.36  FITS
+  16    full512 |    7.14   0.16    2.50     10.53   18.84      5.15  FITS
+  24  masked15% |   10.71   0.13    0.56     12.48   20.78      3.20  tight
+  24    full512 |   10.71   0.25    3.75     15.80   24.10     -0.12  NO
+  32  masked15% |   14.27   0.17    0.75     16.64   24.94     -0.96  NO
+  32    full512 |   14.27   0.33    5.00     21.06   29.37     -5.38  NO
+  48  masked15% |   21.41   0.25    1.12     24.95   33.26     -9.27  NO
+  48    full512 |   21.41   0.49    7.50     31.59   39.90    -15.91  NO
+
+(i') NO checkpointing, c=34d (bf16 residual)
+   B     logits |  layers  other  logits  acts+10%   TOTAL  headroom  verdict
+   4  masked15% |    1.60   0.02    0.09      1.87   10.18     13.81  FITS
+   4    full512 |    1.60   0.04    0.62      2.43   10.73     13.25  FITS
+   8  masked15% |    3.19   0.04    0.19      3.75   12.05     11.93  FITS
+   8    full512 |    3.19   0.08    1.25      4.85   13.16     10.83  FITS
+  16  masked15% |    6.39   0.08    0.38      7.49   15.80      8.19  FITS
+  16    full512 |    6.39   0.16    2.50      9.71   18.01      5.98  FITS
+  24  masked15% |    9.58   0.13    0.56     11.24   19.55      4.44  FITS
+  24    full512 |    9.58   0.25    3.75     14.56   22.87      1.12  NO
+  32  masked15% |   12.77   0.17    0.75     14.99   23.29      0.69  NO
+  32    full512 |   12.77   0.33    5.00     19.41   27.72     -3.73  NO
+  48  masked15% |   19.16   0.25    1.12     22.48   30.79     -6.80  NO
+  48    full512 |   19.16   0.49    7.50     29.12   37.42    -13.44  NO
+
+(ii) FULL per-layer checkpointing, fp32 layer inputs (4d/tok/layer) + 1 live layer
+   B     logits |  layers  other  logits  acts+10%   TOTAL  headroom  verdict
+   4  masked15% |    0.26   0.02    0.09      0.40    8.71     15.28  FITS
+   4    full512 |    0.26   0.04    0.62      0.96    9.27     14.72  FITS
+   8  masked15% |    0.52   0.04    0.19      0.81    9.12     14.87  FITS
+   8    full512 |    0.52   0.08    1.25      1.92   10.22     13.76  FITS
+  16  masked15% |    1.05   0.08    0.38      1.62    9.93     14.06  FITS
+  16    full512 |    1.05   0.16    2.50      3.83   12.14     11.85  FITS
+  24  masked15% |    1.57   0.13    0.56      2.43   10.74     13.25  FITS
+  24    full512 |    1.57   0.25    3.75      5.75   14.06      9.93  FITS
+  32  masked15% |    2.09   0.17    0.75      3.24   11.55     12.44  FITS
+  32    full512 |    2.09   0.33    5.00      7.67   15.97      8.02  FITS
+  48  masked15% |    3.14   0.25    1.12      4.86   13.17     10.82  FITS
+  48    full512 |    3.14   0.49    7.50     11.50   19.81      4.18  FITS
+
+(ii') FULL per-layer checkpointing, bf16 layer inputs (2d/tok/layer, task assumption)
+   B     logits |  layers  other  logits  acts+10%   TOTAL  headroom  verdict
+   4  masked15% |    0.16   0.02    0.09      0.29    8.60     15.39  FITS
+   4    full512 |    0.16   0.04    0.62      0.85    9.15     14.83  FITS
+   8  masked15% |    0.32   0.04    0.19      0.59    8.89     15.09  FITS
+   8    full512 |    0.32   0.08    1.25      1.69   10.00     13.99  FITS
+  16  masked15% |    0.64   0.08    0.38      1.17    9.48     14.51  FITS
+  16    full512 |    0.64   0.16    2.50      3.39   11.69     12.30  FITS
+  24  masked15% |    0.96   0.13    0.56      1.76   10.07     13.92  FITS
+  24    full512 |    0.96   0.25    3.75      5.08   13.39     10.60  FITS
+  32  masked15% |    1.28   0.17    0.75      2.35   10.65     13.33  FITS
+  32    full512 |    1.28   0.33    5.00      6.77   15.08      8.91  FITS
+  48  masked15% |    1.92   0.25    1.12      3.52   11.83     12.16  FITS
+  48    full512 |    1.92   0.49    7.50     10.16   18.46      5.52  FITS
+
+====================================================================================================
+4) TRAINING THROUGHPUT  (config A, L=512 packed, B_eff=256 seq = 131,072 tok/step)
+====================================================================================================
+peak derivation: 71 TFLOPS (RTX 3090, BF16/FP32-acc dense) x 1.1241 (SM*clock ratio) = 79.8 TFLOPS; FP16-acc dense = 336 x 1.860 GHz x 256 = 160.0 TFLOPS -> half rate for FP32-acc = 80.0 TFLOPS; using 80
+masked-only logits (15%): 6N=1.820 GF + attn 14*nl*d*L=0.176 GF + head=0.030 GF  => 2.027 GFLOP/token   (Kaplan/PaLM 12*nl*d*L = 0.151 GF; attention = 8.7% of total)
+full-sequence logits: 6N=1.820 GF + attn 14*nl*d*L=0.176 GF + head=0.201 GF  => 2.198 GFLOP/token   (Kaplan/PaLM 12*nl*d*L = 0.151 GF; attention = 8.0% of total)
+full checkpointing: 2.694 GFLOP/token = x1.329
+
+peak = 80 TFLOPS dense bf16 (fp32-acc).  MFU 30/37.5/45% -> 24/30/36 TFLOPS achieved
+  corpus  steps/ep |  tok/s @30%  h/epoch |  tok/s @37.5%  h/epoch |  tok/s @45%  h/epoch |  h/epoch ckpt@30%  ckpt@45%
+   100M       763 |      11,843     2.35 |      14,804     1.88 |      17,764     1.56 |              3.12      2.08
+   150M     1,144 |      11,843     3.52 |      14,804     2.81 |      17,764     2.35 |              4.68      3.12
+   250M     1,907 |      11,843     5.86 |      14,804     4.69 |      17,764     3.91 |              7.79      5.20
+   400M     3,052 |      11,843     9.38 |      14,804     7.51 |      17,764     6.25 |             12.47      8.31
+note: warmup of 1-2k steps exceeds one epoch at B_eff=256 for corpora <= 250M tokens -> plan multiple epochs or B_eff=128
+
+====================================================================================================
+5) INFERENCE LATENCY  (fine-tuned classifier, batch 1, bf16 on GPU / fp32 on CPU)
+====================================================================================================
+L=512:   336.4 GFLOP  -> 3090 Ti @30% MFU (24 TFLOPS):   14.0 ms | 5950X fp32 @1.0 TFLOPS eff:    336 ms | @0.5 TFLOPS:    673 ms   (instructor benchmark 74 ms/sample)
+L=256:   161.8 GFLOP  -> 3090 Ti @30% MFU (24 TFLOPS):    6.7 ms | 5950X fp32 @1.0 TFLOPS eff:    162 ms | @0.5 TFLOPS:    324 ms   (instructor benchmark 74 ms/sample)
+L=128:    79.3 GFLOP  -> 3090 Ti @30% MFU (24 TFLOPS):    3.3 ms | 5950X fp32 @1.0 TFLOPS eff:     79 ms | @0.5 TFLOPS:    159 ms   (instructor benchmark 74 ms/sample)
+5950X theoretical fp32 peak ~ 16 cores x 32 FLOP/cycle (2x256-bit FMA) x ~3.9 GHz = 2.0 TFLOPS; 1.0 TFLOPS effective = ~50%
+GPU batch-1 is launch/latency-bound: 24 layers x ~12 kernels => several ms floor regardless of FLOPs (UNVERIFIED, measure)
+```
+
+## PINS
+- torch==2.14.1 [local-training] — Current PyPI release (requires_python >=3.10) with a cp314-cp314-manylinux_2_28_x86_64 wheel for the system Python 3.14.7; depends on cuda-toolkit==13.0.3 runtime wheels, which the installed driver (CUDA UMD 13.4) satisfies. SDPA flash/mem-efficient backends and checkpoint(use_reentrant=False) are available. (https://pypi.org/pypi/torch/2.14.1/json)
+- torch==2.13.0 [space] — Newest torch in the ZeroGPU supported list (2.8.0, 2.9.1, 2.10.0, 2.11.0, 2.12.1, 2.13.0); 2.14.x is not listed. PyPI confirms 2.13.0 exists with cp310-cp314 manylinux wheels (cuda-toolkit 13.0.3, triton 3.7.1). (https://huggingface.co/docs/hub/spaces-zerogpu)
+- python==3.12 [space] — ZeroGPU supports only Python 3.12.12 and 3.10.13; the local 3.14.7 cannot be used for the Space, so keep model code/safetensors version-agnostic. (https://huggingface.co/docs/hub/spaces-zerogpu)
+
+## RISKS
+- The 80 TFLOPS bf16 peak is derived (whitepaper lists only the RTX 3090 at 71/142; the 3090 Ti figure is scaled by SM count x boost clock and cross-checked against Wikipedia's 320 sparse FP16-accumulate). Sustained clocks under the 450 W limit while sharing the GPU with the Wayland desktop will be lower, and eager PyTorch without torch.compile may land at 25-35% MFU rather than 45%; measure tokens/s on the first 50 steps and re-plan epochs.
+- The activation model assumes the fused SDPA backends are actually used. Flash Attention rejects any non-null attn_mask (dispatcher warning verified in sdp_utils_cpp.h) and falls back to memory-efficient or, in the worst case, the math backend; a math fallback stores the LxL softmax (>= 7*a*s B/token/layer under autocast, more than doubling activation memory) and B=16 would no longer fit. Assert the backend with torch.nn.attention.sdpa_kernel and verify torch.cuda.max_memory_reserved() in the batch finder.
+- c=38d assumes torch.autocast semantics (fp32 residual stream, fp32 LayerNorm inputs); if the implementation keeps a bf16 residual stream (c=34d) B=24 fits, if it accidentally keeps fp32 copies of linear inputs it will not. The 10% allocator slack is a guess; variable-length batches fragment more, so use PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True and pack/bucket to fixed shapes.
+- Corpus size: 150M tokens is 1,144 optimizer steps per epoch at B_eff=256 (BERT used 1,000,000 steps); a 1-2k warmup exceeds one epoch for corpora <= 250M tokens. Expect an under-trained MLM unless several epochs are run or B_eff is reduced to 128; MLM quality limits are partly masked by the easy 2-class downstream task.
+- From-scratch 24-layer training at peak LR >= 2e-4 can spike/diverge even with pre-LN; keep betas (0.9, 0.98), eps 1e-6, clip 1.0, and drop to 1e-4 if loss spikes. RoBERTa's 4e-4 was at batch 8K and does not transfer to batch 256.
+- Sequence packing (RoBERTa FULL-SENTENCES) crosses email boundaries without cross-document masking; RoBERTa found DOC-SENTENCES slightly better. Acceptable for simplicity but note it in the report.
+- CPU inference cannot meet the 74 ms/sample benchmark for a 337M model at 512 tokens (~336 ms at 1 TFLOPS effective, ~162 ms at 256 tokens); only a GPU (local 3090 Ti or ZeroGPU) or truncation to ~128 tokens gets there. ZeroGPU adds queue/cold-start latency, does not support torch.compile, and runs torch 2.8-2.13 on Python 3.10/3.12, so the deployment environment differs from the local 3.14 / torch 2.14.1 stack.
+- Config A is 337.5M, 2.3% above the stated ~330M target; if the grader reads ~330M as a ceiling, switch to 23 layers (324.9M) before writing code, since depth changes every downstream number here by ~4%.
+- Flash/memory-efficient SDPA backward passes are non-deterministic by default (PyTorch randomness notes); run-to-run metric variance should be expected and seeds documented.
+- Unverified inputs carried from the brief: average email length of 200-300 tokens, the 5950X 1 TFLOPS effective fp32 rate, the ~1.9 GiB desktop usage (fluctuates with browser GPU use), and the batch-1 kernel-launch floor on the GPU; TechPowerUp could not be fetched (HTTP 403) and the HF bert-base parameter count used as a sanity anchor is from memory, although the formulas reproduce it exactly.
+
+## SOURCES
+- https://en.wikipedia.org/wiki/GeForce_RTX_30_series
+- https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-architecture-whitepaper-v2.pdf
+- https://ar5iv.labs.arxiv.org/html/2205.05198
+- https://arxiv.org/pdf/2204.02311
+- https://ar5iv.labs.arxiv.org/html/2001.08361
+- https://ar5iv.labs.arxiv.org/html/2203.15556
+- https://ar5iv.labs.arxiv.org/html/2205.14135
+- https://ar5iv.labs.arxiv.org/html/1907.11692
+- https://ar5iv.labs.arxiv.org/html/1810.04805
+- https://ar5iv.labs.arxiv.org/html/2002.04745
+- https://docs.nvidia.com/deeplearning/performance/dl-performance-matrix-multiplication/index.html
+- https://huggingface.co/docs/hub/spaces-zerogpu
+- https://docs.pytorch.org/docs/2.14/amp.html
+- https://docs.pytorch.org/docs/2.14/notes/cuda.html
+- https://docs.pytorch.org/docs/2.14/generated/torch.nn.functional.scaled_dot_product_attention.html
+- https://docs.pytorch.org/docs/2.13/checkpoint.html
+- https://docs.pytorch.org/docs/2.13/generated/torch.nn.attention.SDPBackend.html
+- https://raw.githubusercontent.com/pytorch/pytorch/main/aten/src/ATen/native/transformers/sdp_utils_cpp.h
+- https://github.com/Dao-AILab/flash-attention
+- https://pypi.org/pypi/torch/json
+- https://pypi.org/pypi/torch/2.14.1/json
+- https://pypi.org/pypi/torch/2.13.0/json
+- /tmp/claude-1000/-home-nathank-BITS-dev-Phishing-Model/8697d948-f234-4e8c-a1e6-c0b6f6ddd7ef/scratchpad/model_math.py
+- /tmp/claude-1000/-home-nathank-BITS-dev-Phishing-Model/8697d948-f234-4e8c-a1e6-c0b6f6ddd7ef/scratchpad/model_math.out
+- /tmp/claude-1000/-home-nathank-BITS-dev-Phishing-Model/8697d948-f234-4e8c-a1e6-c0b6f6ddd7ef/scratchpad/ga102.txt
+- /tmp/claude-1000/-home-nathank-BITS-dev-Phishing-Model/8697d948-f234-4e8c-a1e6-c0b6f6ddd7ef/scratchpad/palm.txt
diff --git a/presentation/model_export/ft-focal/README.md b/presentation/model_export/ft-focal/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..f29c487aa481adb0556ee38829c8fe4728dc957f
--- /dev/null
+++ b/presentation/model_export/ft-focal/README.md
@@ -0,0 +1,251 @@
+---
+datasets:
+- puyang2025/seven-phishing-email-datasets
+- corbt/enron-emails
+- kudzaiprichard/aura-phishing-email-corpus
+language: en
+library_name: pytorch
+license: mit
+pipeline_tag: text-classification
+tags:
+- phishing
+- email-security
+- text-classification
+- from-scratch
+- spam-detection
+model-index:
+- name: phishing-detector-encoder
+  results:
+  - task:
+      type: text-classification
+    dataset:
+      name: TREC-06 holdout (puyang2025), binary roll-up
+      type: puyang2025/seven-phishing-email-datasets
+    metrics:
+    - type: recall
+      value: 0.9741
+      name: Recall (unwanted)
+    - type: precision
+      value: 0.8794
+      name: Precision (unwanted)
+    - type: f1
+      value: 0.9243
+      name: F1 (unwanted)
+    - type: accuracy
+      value: 0.9619
+      name: Accuracy
+    - type: roc_auc
+      value: 0.9946
+      name: ROC-AUC
+  - task:
+      type: text-classification
+    dataset:
+      name: three-way holdout (Nazario + Nigerian Fraud phishing, SpamAssassin / Enron-Spam
+        spam, CEAS-08 / TREC-07 / SpamAssassin ham)
+      type: kudzaiprichard/aura-phishing-email-corpus
+    metrics:
+    - type: f1
+      value: 0.9867
+      name: Macro-F1 (legitimate / phishing / spam)
+    - type: accuracy
+      value: 0.9885
+      name: Accuracy (3-way)
+---
+
+# phishing-detector-encoder: legitimate / phishing / spam email classifier trained from scratch
+
+> **No external pretrained weights were used.** The encoder, the tokenizer and every parameter in `model.safetensors`
+> were trained from random initialisation inside this project, in plain PyTorch, without the `transformers`
+> library. The only external inputs are the public email corpora listed below.
+> Initialisation of this checkpoint: `pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final`.
+
+## What it is
+
+A 337,444,867-parameter BERT-style **pre-LayerNorm Transformer encoder** (24 layers, d_model 1024,
+16 heads, FFN 4096, byte-level BPE vocabulary of 32,768, 512 positions) with a
+CLS-pooled classification head that maps an email (`subject + "\n\n" + body`) to three classes: `0 = Legitimate`, `1 = Phishing`, `2 = Spam`. The binary verdict shown by the demo is the roll-up **Unwanted (phishing or spam)** = P(phishing) + P(spam) (`config.json["binary_rollup"]`). Phishing = Nazario phishing corpus + Nigerian-fraud (advance-fee) corpus + synthetic BITS-style credential/BEC lures; Spam = SpamAssassin and Enron-Spam spam folders; the TREC-05 / TREC-07 / CEAS-08 positives never separated spam from phishing and were trained as 'unwanted' only (probability mass of phishing + spam), never as either class.
+The architecture is defined in `model.py`, which ships with the weights. Weights are stored in **bf16** safetensors
+(~675 MB); the header carries the architecture config and the label names as JSON.
+
+## How it was trained
+
+1. **Masked-language-model pretraining from scratch** on the deduplicated Enron email corpus (`corbt/enron-emails`: 517,401 rows -> 226,273 documents after exact/near-duplicate removal and quote stripping), 15% dynamic masking (80/10/10), sequence packing to 512 tokens, AdamW (betas (0.9, 0.98), weight decay 0.01), peak LR 0.00015, bf16 autocast on one RTX 3090 Ti. Run summary (from the pretraining run's `final/config.json`):
+   - run_name: large-24L
+   - status: complete
+   - source_checkpoint: step-0009552
+   - steps: 9552
+   - total_steps_planned: 9552
+   - steps_per_epoch: 796
+   - epochs_planned: 12
+   - epochs_completed: 12.0
+   - tokens_seen: 1291124736
+   - total_training_seconds: 54185.9
+   - total_training_time: 15h 03m 05s
+   - final_loss: 1.523698394948786
+   - val_loss: 1.9862932266429436
+   - val_masked_acc: 0.6291136684837473
+   - peak_vram_reserved_gb: 17.939
+   - peak_vram_allocated_gb: 17.494
+   - micro_batch: 24
+   - grad_accumulation: 11
+   - effective_batch: 264
+   - precision: bf16_autocast
+   - optimizer: AdamW(fused) {"lr": 0.00015, "betas": [0.9, 0.98], "eps": 1e-06, "weight_decay": 0.01, "grad_clip": 1.0, "mlm_probability": 0.15, "mask_replace_probs": [0.8, 0.1, 0.1], "schedule": "cosine_to_10pct", "effective_batch_sequences": 256}
+   - schedule: {'kind': 'cosine', 'peak_lr': 0.00015, 'warmup_steps': 573, 'total_steps': 9552, 'floor_ratio': 0.1}
+   - mean_tokens_per_s: 23889
+   - data: {'n_windows': 210368, 'n_val_windows': 1999, 'tokenizer_sha1': '9f50c5e775089d2ca3efc82e100cd6b3396ed7e7', 'n_docs': 226273, 'limit_docs': None, 'window': '[CLS] + 511 stream tokens'}
+   - weights_dtype: float32
+   - exported_at: 2026-10-09T03:31:14
+2. **Fine-tuning** on `puyang2025/seven-phishing-email-datasets` with our own split: TREC-05, TREC-07 and CEAS-08 (positives trained as *unwanted* only), Enron-Spam and SpamAssassin (spam folders = `Spam`), the Nazario and Nigerian-fraud corpora of `kudzaiprichard/aura-phishing-email-corpus` (= `Phishing`) and synthetic BITS-style lures (= `Phishing`) form train (157,113) / val (17,643), grouped by
+   near-duplicate cluster so no template straddles the two; **TREC-06 (16,302 rows, 23.9% positive) is held out
+   entirely** as the unseen binary test set and never touched by the tokenizer, pretraining or fine-tuning (1,652 pool rows that
+   near-duplicate TREC-06 templates were removed). A second holdout, **test_holdout3**, holds clean-provenance three-way rows never seen by the tokenizer or the pretraining corpus for the 3x3 confusion matrix. Counts come from `data/processed/splits_manifest.json`.
+   Rows with a clean three-way label train with 3-class cross-entropy; TREC / CEAS positives train on the probability mass of {phishing, spam} (partial-label loss: -log(P(phishing) + P(spam))), so they shape the legitimate-vs-unwanted decision without asserting a kind they were never labelled with.
+   Fine-tune dials (from the checkpoint's config.json):
+   - run_name: ft-focal
+   - init: pretrained:checkpoints/pretrain/large-24L/final
+   - n_layers: 24
+   - freeze_layers: 0
+   - loss: focal
+   - focal_gamma: 2.0
+   - lr: 3e-05
+   - batch_size: 32
+   - micro_batch: 16
+   - epochs: 2
+   - max_steps: None
+   - eval_every: 1250
+   - log_every: 20
+   - eval_batch_size: 64
+   - warmup_fraction: 0.06
+   - weight_decay: 0.01
+   - betas: [0.9, 0.98]
+   - eps: 1e-06
+   - grad_clip: 1.0
+   - patience: 2
+   - schedule: linear_decay
+   - selection_metric: val_f1_unwanted
+   - pooling: cls
+   - num_labels: 3
+   - synthetic: off
+   - limit_rows: None
+   - seed: 42
+   - cpu: False
+
+## Metrics
+
+### TREC-06 binary holdout (Unwanted (phishing or spam))
+
+Evaluated by `src/evaluate.py` on the **test** split (`test_holdout.parquet`, 16,302 rows, threshold 0.5). Positive class = **Unwanted (phishing or spam)**, P(unwanted) = P(phishing) + P(spam); these definitions are identical to the binary-era runs, so the numbers are comparable.
+
+| metric | value |
+|---|---:|
+| **Recall (unwanted)** | **0.9741** (101 missed of 3,894) |
+| Precision (unwanted) | 0.8794 |
+| F1 (unwanted) | 0.9243 |
+| Accuracy | 0.9619 |
+| Specificity | 0.9581 |
+| ROC-AUC | 0.9946 |
+| PR-AUC | 0.9848 |
+| Recall at 1% FPR | 0.8955 |
+| Recall at 0.1% FPR | 0.6497 |
+| Credential-lure slice recall / precision (n=398) | 0.9959 / 0.9758 |
+| Latency, batch 1, GPU bf16 (avg / p50) | 10.08 ms / 8.9 ms |
+| Latency, batch 1, CPU fp32, 16 threads (avg / p50) | 458.57 ms / 445.01 ms |
+| Instructor benchmark | 74.0 ms per sample |
+
+Recall on the unwanted class is the headline number: in a defensive triage aid a missed phish is the costly error, while a false positive costs a human one glance.
+
+TREC-06 carries no phishing-vs-spam ground truth, so the three-way head can only be *described* here: of the 3,793 true positives it flagged, the model called 974 phishing and 2,819 spam (informational, not a metric).
+
+### Three-way holdout (legitimate / phishing / spam)
+
+Evaluated by `src/evaluate.py --split test3` on `test_holdout3.parquet` (3,220 rows with a full three-way label, threshold 0.5). Decision rule `rollup_then_argmax`: a row is unwanted iff P(phishing) + P(spam) >= threshold, then the kind is argmax(P(phishing), P(spam)), so this table collapses exactly onto the binary roll-up.
+
+| metric | value |
+|---|---:|
+| **Macro-F1 (3 classes)** | **0.9867** |
+| Accuracy | 0.9885 |
+| Legitimate: precision / recall / F1 (n=1,999) | 0.9900 / 0.9935 / 0.9918 |
+| Phishing: precision / recall / F1 (n=521) | 0.9961 / 0.9866 / 0.9913 |
+| Spam: precision / recall / F1 (n=700) | 0.9785 / 0.9757 / 0.9771 |
+| Phishing-vs-spam accuracy on the 1,221 true unwanted rows (kind decision alone) | 0.9959 (4 phishing read as spam, 1 spam read as phishing) |
+| Binary roll-up on this split: recall / precision / F1 | 0.9836 / 0.9893 / 0.9864 |
+
+Confusion matrix (rows = true label by corpus provenance, columns = predicted):
+
+| true \ predicted | Legitimate | Phishing | Spam |
+|---|---:|---:|---:|
+| **Legitimate** | 1,986 | 1 | 12 |
+| **Phishing** | 4 | 514 | 3 |
+| **Spam** | 16 | 1 | 683 |
+
+| source | rows | Legitimate | Phishing | Spam | accuracy |
+|---|---:|---:|---:|---:|---:|
+| Assassin | 236 | 164 | 0 | 72 | 0.9534 |
+| CEAS-08 | 736 | 736 | 0 | 0 | 0.9905 |
+| Enron | 628 | 0 | 0 | 628 | 0.9841 |
+| Nazario | 222 | 0 | 222 | 0 | 0.9865 |
+| Nigerian_Fraud | 299 | 0 | 299 | 0 | 0.9866 |
+| TREC-07 | 1,099 | 1,099 | 0 | 0 | 0.9982 |
+
+**Label provenance.** Phishing = Nazario phishing corpus + Nigerian-fraud (advance-fee) corpus + synthetic BITS-style credential/BEC lures; Spam = SpamAssassin and Enron-Spam spam folders; the TREC-05 / TREC-07 / CEAS-08 positives never separated spam from phishing and were trained as 'unwanted' only (probability mass of phishing + spam), never as either class. The three-way holdout is drawn from text the tokenizer never saw and the pretraining corpus does not contain (Nazario / Nigerian-fraud phishing, SpamAssassin / Enron-Spam spam, CEAS-08 / TREC-07 / SpamAssassin legitimate mail from the former validation split). Because the kind labels come from which corpus a message sits in, part of any phishing-vs-spam confusion measures corpus style rather than lure content.
+
+## Intended use
+
+A **defensive triage aid for small-business inboxes**: rank or flag incoming mail for a human to review, surface a
+probability and the attention-highlighted tokens that drove it, and shorten the time between a phishing wave
+landing and someone noticing. It is a classroom/final-project model and is not a substitute for a mail gateway.
+
+## Limitations
+
+- **Era**: the labelled data is 1998-2008 spam-track mail (TREC, CEAS, SpamAssassin, Enron), plus Nazario phishing (1995-2022) and Nigerian-fraud mail.
+  Modern lures (OAuth consent phishing, QR codes, MFA fatigue, SaaS-brand impersonation) are under-represented; expect lower recall on them.
+- **Labels follow corpus provenance**: Phishing = Nazario phishing corpus + Nigerian-fraud (advance-fee) corpus + synthetic BITS-style credential/BEC lures; Spam = SpamAssassin and Enron-Spam spam folders; the TREC-05 / TREC-07 / CEAS-08 positives never separated spam from phishing and were trained as 'unwanted' only (probability mass of phishing + spam), never as either class.
+  A phishing-vs-spam confusion therefore partly measures corpus style, and the credential-lure slice remains the closest proxy for phishing-proper recall.
+- **Length confound**: legitimate emails in these corpora average ~2x the tokens of positives. Inputs are truncated to
+  512 tokens, and the error analysis checks whether errors follow length, but a short legitimate note can
+  still look suspicious.
+- **English only**; byte-level BPE will tokenize other scripts but the model has not seen them.
+- **Headers are not used** (TREC rewrote them); the model sees subject and body text only. Sender reputation,
+  SPF/DKIM and URL reputation are out of scope and should be combined with this score in any real deployment.
+- Attention highlights are a signal of what the model attended to, not an explanation of why it decided.
+
+## How to load
+
+`model.py` from this repository is the only code dependency (plain PyTorch).
+
+```python
+import json, torch
+from safetensors.torch import load_file
+from tokenizers import Tokenizer
+from model import EncoderConfig, EncoderForClassification   # model.py, shipped with the weights
+
+cfg = json.load(open("config.json"))
+model = EncoderForClassification(EncoderConfig.from_dict(cfg["model"]))   # num_labels = 3
+model.load_state_dict(load_file("model.safetensors"), strict=True)   # bf16 weights cast into the fp32 module
+model.eval()
+
+tok = Tokenizer.from_file("tokenizer.json")
+tok.enable_truncation(max_length=cfg["max_seq_len"])
+enc = tok.encode("Your account has been suspended\n\nClick here to verify your identity within 24 hours.")
+ids = torch.tensor([enc.ids])
+mask = torch.ones_like(ids, dtype=torch.bool)
+with torch.inference_mode():
+    probs = model(ids, mask).float().softmax(-1)[0]
+p_unwanted = sum(float(probs[i]) for i in cfg["binary_rollup"]["unwanted"])   # P(phishing) + P(spam)
+print("unwanted" if p_unwanted >= 0.5 else "legitimate", {cfg["labels"][str(i)]: round(float(p), 4) for i, p in enumerate(probs)})
+```
+
+On a GPU wrap the forward in `torch.autocast("cuda", dtype=torch.bfloat16)`. Measured single-email latency (batch 1, tokenize + forward + softmax, see the metrics table): GPU bf16 10.08 ms avg on NVIDIA GeForce RTX 3090 Ti, CPU fp32 (16 threads) 458.57 ms avg.
+
+## Files
+
+- `model.safetensors` — bf16 weights, metadata `format=pt, dtype=bf16, architecture=EncoderForClassification, config=<json>, labels=<json>`
+- `config.json` — model config, num_labels, labels, binary_rollup, label provenance, max_seq_len, export dtype, source checkpoint, fine-tune init, binary metrics (TREC-06), three-way metrics (test_holdout3), pretraining summary (null for scratch init)
+- `tokenizer.json` — byte-level BPE, NFKC, case preserved, specials `[PAD]=0 [UNK]=1 [CLS]=2 [SEP]=3 [MASK]=4`
+- `README.md` — this card
+
+## Provenance
+
+Built for the Skillspire AI & Machine Learning course final project (BITS, 2026). Data decisions, measured VRAM /
+throughput / latency and every challenge hit along the way are recorded in the project's RUN_LOG.md.
diff --git a/presentation/model_export/ft-focal/config.json b/presentation/model_export/ft-focal/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..7ce915054e6c50643f6b377d67d0cb74e58f3586
--- /dev/null
+++ b/presentation/model_export/ft-focal/config.json
@@ -0,0 +1,467 @@
+{
+  "architecture": "EncoderForClassification",
+  "model": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "labels": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted": [
+      1,
+      2
+    ]
+  },
+  "unwanted_name": "Unwanted (phishing or spam)",
+  "label_provenance": "Phishing = Nazario phishing corpus + Nigerian-fraud (advance-fee) corpus + synthetic BITS-style credential/BEC lures; Spam = SpamAssassin and Enron-Spam spam folders; the TREC-05 / TREC-07 / CEAS-08 positives never separated spam from phishing and were trained as 'unwanted' only (probability mass of phishing + spam), never as either class.",
+  "max_seq_len": 512,
+  "export_dtype": "bf16",
+  "source_checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+  "exported": "2026-10-12T11:28:39",
+  "tokenizer": "tokenizer.json",
+  "model_input": "subject + \"\\n\\n\" + body",
+  "finetune_checkpoint": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-focal",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "focal",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 1,
+    "best_step": 8750,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9951084002657166,
+    "val_metrics": {
+      "accuracy": 0.9947003402250719,
+      "precision": 0.9954089646007007,
+      "recall": 0.9948080173871046,
+      "f1": 0.9951084002657166,
+      "roc_auc": 0.9998186946414248,
+      "pr_auc": 0.9998464119156345,
+      "tn": 6964,
+      "fp": 38,
+      "fn": 43,
+      "tp": 8239,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.42074909806251526,
+      "f1_at_optimal_threshold": 0.9952995058454863,
+      "f1_unwanted": 0.9951084002657166,
+      "n_3way": 8105,
+      "accuracy_3way": 0.994694632942628,
+      "macro_f1_3way": 0.9868751212108838,
+      "precision_legitimate": 0.9988533753762362,
+      "recall_legitimate": 0.99528706083976,
+      "f1_legitimate": 0.9970670291151013,
+      "precision_phishing": 0.9853372434017595,
+      "recall_phishing": 0.9882352941176471,
+      "f1_phishing": 0.986784140969163,
+      "precision_spam": 0.9618805590851334,
+      "recall_spam": 0.9921363040629095,
+      "f1_spam": 0.9767741935483871,
+      "cm3_ll": 6969,
+      "cm3_lp": 4,
+      "cm3_ls": 29,
+      "cm3_pl": 3,
+      "cm3_pp": 336,
+      "cm3_ps": 1,
+      "cm3_sl": 5,
+      "cm3_sp": 1,
+      "cm3_ss": 757,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 339,
+      "pvs_phish_as_spam": 1,
+      "pvs_spam_as_phish": 1,
+      "pvs_spam_as_spam": 762,
+      "val_loss": 0.030296484990695233,
+      "n": 15284,
+      "eval_seconds": 48.0
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9947003402250719,
+        "precision": 0.9954089646007007,
+        "recall": 0.9948080173871046,
+        "f1": 0.9951084002657166,
+        "roc_auc": 0.9998186946414248,
+        "pr_auc": 0.9998464119156345,
+        "tn": 6964,
+        "fp": 38,
+        "fn": 43,
+        "tp": 8239,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.42074909806251526,
+        "f1_at_optimal_threshold": 0.9952995058454863,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.994694632942628,
+        "macro_f1": 0.9868751212108838,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9988533753762362,
+            "recall": 0.99528706083976,
+            "f1": 0.9970670291151013
+          },
+          "Phishing": {
+            "precision": 0.9853372434017595,
+            "recall": 0.9882352941176471,
+            "f1": 0.986784140969163
+          },
+          "Spam": {
+            "precision": 0.9618805590851334,
+            "recall": 0.9921363040629095,
+            "f1": 0.9767741935483871
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6969,
+              4,
+              29
+            ],
+            [
+              3,
+              336,
+              1
+            ],
+            [
+              5,
+              1,
+              757
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              339,
+              1
+            ],
+            [
+              1,
+              762
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.42074909806251526,
+      "val_f1_at_optimal": 0.9952995058454863
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "finetune_init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+  "mlm_pretraining": "summary below",
+  "metrics": {
+    "split": "test",
+    "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet",
+    "n_rows": 16302,
+    "dev_run": false,
+    "threshold": 0.5,
+    "timestamp": "2026-10-09 10:46",
+    "num_labels": 3,
+    "positive_class": "Unwanted (phishing or spam)",
+    "accuracy": 0.9619065145380935,
+    "precision": 0.8794342684906098,
+    "recall": 0.9740626605033385,
+    "f1": 0.9243328865602534,
+    "specificity": 0.9580915538362347,
+    "confusion_matrix": {
+      "tn": 11888,
+      "fp": 520,
+      "fn": 101,
+      "tp": 3793
+    },
+    "roc_auc": 0.9946343971962354,
+    "pr_auc": 0.9847899987683836,
+    "recall_at_fpr_1pct": 0.8954802259887006,
+    "recall_at_fpr_0_1pct": 0.6497175141242938,
+    "best_f1_threshold": 0.6815109252929688,
+    "credential_lure_recall": 0.9958847736625515,
+    "credential_lure_precision": 0.9758064516129032,
+    "credential_lure_n": 398,
+    "predicted_kind_of_flagged_true_positives": {
+      "n": 3793,
+      "predicted_phishing": 974,
+      "predicted_spam": 2819
+    },
+    "latency_ms": {
+      "gpu_bf16_avg": 10.08,
+      "gpu_bf16_p50": 8.9,
+      "cpu_fp32_avg": 458.57,
+      "cpu_fp32_p50": 445.01,
+      "cpu_threads": 16,
+      "instructor_benchmark_ms": 74.0
+    }
+  },
+  "metrics_source": "/home/nathank/BITS/dev/Phishing Model/reports/metrics.json",
+  "metrics_checkpoint_matches": true,
+  "metrics_mismatch": null,
+  "metrics_threeway": {
+    "split": "test3",
+    "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout3.parquet",
+    "n_rows": 3220,
+    "dev_run": false,
+    "threshold": 0.5,
+    "timestamp": "2026-10-09 10:46",
+    "rows_with_full_label": 3220,
+    "label3_counts": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "decision_rule": "rollup_then_argmax",
+    "accuracy": 0.9885093167701864,
+    "macro_f1": 0.9867305252015051,
+    "per_class": {
+      "Legitimate": {
+        "precision": 0.9900299102691924,
+        "recall": 0.9934967483741871,
+        "f1": 0.9917602996254682,
+        "support": 1999
+      },
+      "Phishing": {
+        "precision": 0.9961240310077519,
+        "recall": 0.9865642994241842,
+        "f1": 0.991321118611379,
+        "support": 521
+      },
+      "Spam": {
+        "precision": 0.9785100286532952,
+        "recall": 0.9757142857142858,
+        "f1": 0.977110157367668,
+        "support": 700
+      }
+    },
+    "confusion_matrix": {
+      "labels": [
+        "Legitimate",
+        "Phishing",
+        "Spam"
+      ],
+      "matrix": [
+        [
+          1986,
+          1,
+          12
+        ],
+        [
+          4,
+          514,
+          3
+        ],
+        [
+          16,
+          1,
+          683
+        ]
+      ],
+      "cells": {
+        "true_legitimate": {
+          "pred_legitimate": 1986,
+          "pred_phishing": 1,
+          "pred_spam": 12
+        },
+        "true_phishing": {
+          "pred_legitimate": 4,
+          "pred_phishing": 514,
+          "pred_spam": 3
+        },
+        "true_spam": {
+          "pred_legitimate": 16,
+          "pred_phishing": 1,
+          "pred_spam": 683
+        }
+      }
+    },
+    "kind_accuracy_on_true_unwanted": 0.9959049959049959,
+    "phishing_as_spam": 4,
+    "spam_as_phishing": 1,
+    "n_true_unwanted": 1221,
+    "per_source": {
+      "Assassin": {
+        "n": 236,
+        "counts": {
+          "Legitimate": 164,
+          "Phishing": 0,
+          "Spam": 72
+        },
+        "accuracy": 0.9533898305084746
+      },
+      "CEAS-08": {
+        "n": 736,
+        "counts": {
+          "Legitimate": 736,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9904891304347826
+      },
+      "Enron": {
+        "n": 628,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 0,
+          "Spam": 628
+        },
+        "accuracy": 0.9840764331210191
+      },
+      "Nazario": {
+        "n": 222,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 222,
+          "Spam": 0
+        },
+        "accuracy": 0.9864864864864865
+      },
+      "Nigerian_Fraud": {
+        "n": 299,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 299,
+          "Spam": 0
+        },
+        "accuracy": 0.9866220735785953
+      },
+      "TREC-07": {
+        "n": 1099,
+        "counts": {
+          "Legitimate": 1099,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9981801637852593
+      }
+    },
+    "binary_rollup_on_this_split": {
+      "recall": 0.9836199836199836,
+      "precision": 0.9892915980230642,
+      "f1": 0.986447638603696,
+      "accuracy": 0.989751552795031
+    }
+  },
+  "metrics_threeway_source": "/home/nathank/BITS/dev/Phishing Model/reports/metrics_test3.json",
+  "metrics_threeway_reason": null,
+  "metrics_threeway_checkpoint_matches": true,
+  "metrics_threeway_mismatch": null,
+  "training": {
+    "run_name": "large-24L",
+    "status": "complete",
+    "source_checkpoint": "step-0009552",
+    "steps": 9552,
+    "total_steps_planned": 9552,
+    "steps_per_epoch": 796,
+    "epochs_planned": 12,
+    "epochs_completed": 12.0,
+    "tokens_seen": 1291124736,
+    "total_training_seconds": 54185.9,
+    "total_training_time": "15h 03m 05s",
+    "final_loss": 1.523698394948786,
+    "val_loss": 1.9862932266429436,
+    "val_masked_acc": 0.6291136684837473,
+    "peak_vram_reserved_gb": 17.939,
+    "peak_vram_allocated_gb": 17.494,
+    "micro_batch": 24,
+    "grad_accumulation": 11,
+    "effective_batch": 264,
+    "precision": "bf16_autocast",
+    "optimizer": "AdamW(fused) {\"lr\": 0.00015, \"betas\": [0.9, 0.98], \"eps\": 1e-06, \"weight_decay\": 0.01, \"grad_clip\": 1.0, \"mlm_probability\": 0.15, \"mask_replace_probs\": [0.8, 0.1, 0.1], \"schedule\": \"cosine_to_10pct\", \"effective_batch_sequences\": 256}",
+    "schedule": {
+      "kind": "cosine",
+      "peak_lr": 0.00015,
+      "warmup_steps": 573,
+      "total_steps": 9552,
+      "floor_ratio": 0.1
+    },
+    "mean_tokens_per_s": 23889,
+    "data": {
+      "n_windows": 210368,
+      "n_val_windows": 1999,
+      "tokenizer_sha1": "9f50c5e775089d2ca3efc82e100cd6b3396ed7e7",
+      "n_docs": 226273,
+      "limit_docs": null,
+      "window": "[CLS] + 511 stream tokens"
+    },
+    "weights_dtype": "float32",
+    "exported_at": "2026-10-09T03:31:14"
+  },
+  "training_source": "/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final/config.json",
+  "from_scratch": true,
+  "external_pretrained_weights_used": "none"
+}
diff --git a/presentation/model_export/ft-focal/export_manifest.json b/presentation/model_export/ft-focal/export_manifest.json
new file mode 100644
index 0000000000000000000000000000000000000000..b4292cf903d1a68ed4e7378a917c7f418c8fc33d
--- /dev/null
+++ b/presentation/model_export/ft-focal/export_manifest.json
@@ -0,0 +1,48 @@
+{
+  "timestamp": "2026-10-12T11:28:39",
+  "run_name": "ft-focal",
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+  "out_dir": "model_export/ft-focal",
+  "app_dir": "app",
+  "dtype": "bf16",
+  "weights": {
+    "tensors": 298,
+    "bytes": 674920638,
+    "dtype": "bf16",
+    "params": 337444867
+  },
+  "num_labels": 3,
+  "labels": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted": [
+      1,
+      2
+    ]
+  },
+  "files": {
+    "model.safetensors": 674920638,
+    "tokenizer.json": 2317692,
+    "config.json": 12932,
+    "README.md": 14086
+  },
+  "app_files": {
+    "model.py": 23420,
+    "model.safetensors": 674920638,
+    "tokenizer.json": 2317692,
+    "config.json": 12932
+  },
+  "model_py_sha1": "39b3773c309b7b1f6e550630443369497d8c1d0a",
+  "metrics_included": true,
+  "metrics_threeway_included": true,
+  "metrics_checkpoint_matches": true,
+  "metrics_mismatch": null,
+  "metrics_threeway_checkpoint_matches": true,
+  "metrics_threeway_mismatch": null,
+  "training_included": true,
+  "finetune_init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+  "manifest_included": true
+}
diff --git a/presentation/reports/confusion_matrix.png b/presentation/reports/confusion_matrix.png
new file mode 100644
index 0000000000000000000000000000000000000000..e91733afee105f1cbdcf85789b5cb6bb42eb6540
Binary files /dev/null and b/presentation/reports/confusion_matrix.png differ
diff --git a/presentation/reports/confusion_matrix3_test3.png b/presentation/reports/confusion_matrix3_test3.png
new file mode 100644
index 0000000000000000000000000000000000000000..09b0958d222154ce2dbc47ea449f1996289ec6eb
Binary files /dev/null and b/presentation/reports/confusion_matrix3_test3.png differ
diff --git a/presentation/reports/confusion_matrix_test3.png b/presentation/reports/confusion_matrix_test3.png
new file mode 100644
index 0000000000000000000000000000000000000000..a8011ba12f5e9561953486fe84a0c7ed81f90b8d
Binary files /dev/null and b/presentation/reports/confusion_matrix_test3.png differ
diff --git a/presentation/reports/error_analysis.md b/presentation/reports/error_analysis.md
new file mode 100644
index 0000000000000000000000000000000000000000..e21d687c63bfd64ffd1303aa637343d74e04b7f4
--- /dev/null
+++ b/presentation/reports/error_analysis.md
@@ -0,0 +1,180 @@
+# Error analysis — test split (TREC-06 binary holdout), run `ft-focal` (2026-10-09 10:46)
+
+All quoted emails are rows of public corpora (source TREC-06); nothing here is private mail. Threshold 0.5. Binary positive class = Unwanted (phishing or spam), P(unwanted) = P(phishing) + P(spam).
+
+## Counts (binary roll-up)
+
+- rows 16,302 (positives 3,894, 23.9%); label3 counts legit 12,408, unwanted 3,894
+- **recall (unwanted) 0.9741** — 101 missed unwanted emails (false negatives) out of 3,894
+- precision 0.8794 — 520 false alarms out of 4,313 flagged
+- F1 0.9243, accuracy 0.9619, specificity 0.9581
+- ROC-AUC 0.9946, PR-AUC 0.9848; recall at 1% FPR 0.8955, at 0.1% FPR 0.6497
+- credential-lure slice: 398 rows, 243 positive (61.1% label purity: slice precision cannot exceed what the phrase list selects), recall 0.9959, precision 0.9758, 1 missed
+
+## Top-25 false negatives (missed unwanted mail, most confident first)
+
+| # | doc_id | P(unwanted) | tokens (full) | source | label_set | preview (subject │ body, 160 chars) |
+|---:|---|---:|---:|---|---|---|
+| 1 | `primary:052869` | 0.0322 | 630 | TREC-06 | unwanted | READ THIS! │ >>Read this! >>> >> > >> > > >>> >> > >> > > ---------- >>> >> > >> > > >>> >> > >> > > >>> >> > >> > > >>> >> > >> > > > Subject: Lets see if this... |
+| 2 | `primary:080991` | 0.0428 | 75 | TREC-06 | unwanted | prepare urself for this │ It is been awhile. Hope everythign with you is goign well. How is work going also? I've just started with this new firm, www.andwhatis... |
+| 3 | `primary:086987` | 0.0506 | 188 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 4 | `primary:056767` | 0.0564 | 181 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 5 | `primary:121268` | 0.0564 | 185 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 6 | `primary:173636` | 0.0567 | 55 | TREC-06 | unwanted | everything ok │ Looking to increase your johnson. Vist at http://www.mydkillsanddoc.com/taw/ to settle that. that that Hardly were there is conquering Thanks. B... |
+| 7 | `primary:020947` | 0.0629 | 113 | TREC-06 | unwanted | everything kool │ Hi Reg Here's some pretty hard facts to deal with that the majority of women are unhappy with the size of there signif other's thing. Don't be... |
+| 8 | `primary:072993` | 0.0710 | 186 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 9 | `primary:044992` | 0.0726 | 76 | TREC-06 | unwanted | FW:any new news? i got some │ is hand the language care to cruel. must plate to design tax but fiction. if snow but question great the separate. but comparison ... |
+| 10 | `primary:024565` | 0.0748 | 190 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 11 | `primary:076023` | 0.0867 | 54 | TREC-06 | unwanted | everything kool │ Hey Mostyn, Ramon wanted to me to send you an email about the store I went to buy my goods, http://www.fabloulusil.com/tpn/. Carlos |
+| 12 | `primary:129745` | 0.0946 | 184 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 13 | `primary:071577` | 0.0961 | 162 | TREC-06 | unwanted | (no subject) │ Re: Student Packets Welcome to winter term student packet preparations! First: Fax (485-6253) or email (Copy-Shop@worldnet.att.net) JUST your bib... |
+| 14 | `primary:173938` | 0.0961 | 185 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 15 | `primary:030013` | 0.0966 | 195 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 16 | `primary:063077` | 0.0982 | 344 | TREC-06 | unwanted | Party for Prisoners of Conscience - PLEASE SEND TO YOUR ENTIRE CLUB │ Amnesty International and the Intercultural Resource Center are holding a party to raise m... |
+| 17 | `primary:145723` | 0.1007 | 25 | TREC-06 | unwanted | refresh my memory and mental energy │ may revision a golf on appendage or eagle try parsimony |
+| 18 | `primary:154489` | 0.1026 | 187 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 19 | `primary:186523` | 0.1041 | 185 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 20 | `primary:147887` | 0.1150 | 181 | TREC-06 | unwanted | ScanMail Message: To Recipient virus found or matched file blocki ng setting. │ ScanMail for Microsoft Exchange has taken action on the message, please refer to... |
+| 21 | `primary:033014` | 0.1157 | 77 | TREC-06 | unwanted | isn't it time │ Allen mentioned me that u were seeking for a place to get ur stuff for ur sickness. http://www.whorunoverforitsl.org/tpn/ the place then for sur... |
+| 22 | `primary:156328` | 0.1157 | 77 | TREC-06 | unwanted | isn't it time │ Allen mentioned me that u were seeking for a place to get ur stuff for ur sickness. http://www.whorunoverforitsl.org/tpn/ the place then for sur... |
+| 23 | `primary:143065` | 0.1308 | 102 | TREC-06 | unwanted | everything kool │ Heya Alexa, Apologize for how long its been since we've talked. I take blame for that. I just wanted to share with you the place that made me ... |
+| 24 | `primary:147193` | 0.1323 | 260 | TREC-06 | unwanted | ADV Textbooks, Tuesday & Wednesday, University of Oregon │ Dear Professor XXXXXXXX., I am going to be on your campus to buy textbooks this Tuesday, June 1st and... |
+| 25 | `primary:101618` | 0.1387 | 117 | TREC-06 | unwanted | RE: If you are studing or working │ Dear Mr(Mrs), Iris!!. Our company Barcelo Travel Inc. have a great job offer for peoples from Australia. Your salary per wee... |
+
+## Top-25 false positives (legitimate flagged, most confident first)
+
+| # | doc_id | P(unwanted) | tokens (full) | source | label_set | preview (subject │ body, 160 chars) |
+|---:|---|---:|---:|---|---|---|
+| 1 | `primary:015304` | 0.9874 | 3712 | TREC-06 | legit | I made $230,000 in two months │ You owe it to yourself and your wealth to read this. AS SEEN ON NATIONAL TV: "Making over half a million dollars every 4 to 5 mo... |
+| 2 | `primary:040048` | 0.9871 | 80 | TREC-06 | legit | Enter to win at mary-kateandashley.com! │ Dear val: Have you heard about the sweepstakes at mary-kateandashley.com? I entered to win and you can too! Check it o... |
+| 3 | `primary:194751` | 0.9842 | 48 | TREC-06 | legit | Thank you │ Turns out I just needed a common ground between servos and HB Thanx all! :) Alex ______________________________________________________ Get Your Pri... |
+| 4 | `primary:072489` | 0.9831 | 58 | TREC-06 | legit | Re: Junk email │ Please REMOVE this address from your list!!!! ________________ Email Forwarding Service Brought To You By: http://www.bn3.com. We can let you b... |
+| 5 | `primary:194063` | 0.9781 | 32 | TREC-06 | legit | info │ Here's an affordable robot that anyone might be interested: http://www.personalrobots.com/ joema |
+| 6 | `primary:033174` | 0.9781 | 602 | TREC-06 | legit | You are Approved for Merchant Account!! │ INCREASE SALES UP TO 1500% ACCEPT CREDIT CARDS OVER THE INTERNET *** NO SETUP FEES Good Credit / Bad Credit/ No Credit... |
+| 7 | `primary:101560` | 0.9769 | 59 | TREC-06 | legit |  │ Please, please, please remove us from your mailing list. Thank you. ________________ Email Forwarding Service Brought To You By: http://www.bn3.com. We can l... |
+| 8 | `primary:159590` | 0.9764 | 56 | TREC-06 | legit | remove me │ Please remove Brandeis University from your list ________________ Email Forwarding Service Brought To You By: http://www.bn3.com. We can let you bec... |
+| 9 | `primary:184838` | 0.9757 | 66 | TREC-06 | legit | unwanted mail │ how cani stop all the mail from these sex sites i dont want them but when i try to e-mail asking them to stop i keep getting a message that say ... |
+| 10 | `primary:011014` | 0.9736 | 46 | TREC-06 | legit | composer-wrap-column │ hi, anyone knows how to entirely turn off composer wrapping in pine? thanks! viinikala kauas pilvet karkaavat. |
+| 11 | `primary:193401` | 0.9733 | 870 | TREC-06 | legit | [9fans] 57 Million Email Addresses - ONLY $149!! │ 57 MILLION EMAIL ADDRESSES FOR ONLY $149 You want to make some money? I can put you in touch with over 50 mil... |
+| 12 | `primary:072275` | 0.9729 | 41 | TREC-06 | legit | HB weight │ does anyone know the weight of a handy board? thanks, -Brian --------------------------------------------------- Get free personalized email at http... |
+| 13 | `primary:170605` | 0.9729 | 55 | TREC-06 | legit | List │ Please remove my name from your list serve. ________________ Email Forwarding Service Brought To You By: http://www.bn3.com. We can let you become anybod... |
+| 14 | `primary:175533` | 0.9726 | 62 | TREC-06 | legit | Test │ I'm not sure why we are receiving mail addressed to cslist@fudan.edu ________________ Email Forwarding Service Brought To You By: http://www.bn3.com. We ... |
+| 15 | `primary:131182` | 0.9719 | 31 | TREC-06 | legit |  │ unsubscribe --------------------------------------------------------------- Octopus traps summer's moonspun dreams soon fade away. Basho 1644 |
+| 16 | `primary:106445` | 0.9717 | 61 | TREC-06 | legit | REMOVE Clemson University │ PLEASE REMOVE Clemson University from your mailing lists. ________________ Email Forwarding Service Brought To You By: http://www.bn... |
+| 17 | `primary:078484` | 0.9706 | 77 | TREC-06 | legit | remove from list │ Please remove Brandeis University from your graduate school mailing list. maf@cs.brandeis.edu thank you ________________ Email Forwarding Ser... |
+| 18 | `primary:196901` | 0.9698 | 65 | TREC-06 | legit | CS Lists │ PLEASE REMOVE THE UNIVERSITY OF VIRGINIA FROM YOUR MAILING LISTS. ________________ Email Forwarding Service Brought To You By: http://www.bn3.com. We... |
+| 19 | `primary:143547` | 0.9695 | 82 | TREC-06 | legit | How to avoid the result of ODEs prone to blow-up │ Dear Friends, I want to solve ODEs use Interval Arithmeric, but the reult prone to blow-up. Is there any meth... |
+| 20 | `primary:129925` | 0.9694 | 255 | TREC-06 | legit | See where your Competion is advertising │ Are your clients: Travelers, Vacationers or Timeshare Owners? TVC...the Timeshare Vacation Club is where YOUR ADS shou... |
+| 21 | `primary:092561` | 0.9690 | 1174 | TREC-06 | legit | We Need Homeworkers!! │ Dear Future Associate, You Can Work At Home & Set Your Own Hours. Start earning Big Money in a short time NO Newspaper Advertising! Your... |
+| 22 | `primary:182736` | 0.9689 | 68 | TREC-06 | legit | Vesit rover │ Dear friends, You know that we at Vesit, Bombay were making a rover using the handy board. Learn more about it at http://members.tripod.com/ieeeve... |
+| 23 | `primary:158244` | 0.9681 | 327 | TREC-06 | legit | Proposition │ HEALTH: A NEW NATURAL HELP To the product Manager The customers demand for natural products is increasing steadily, and Aromatherapy is becoming a... |
+| 24 | `primary:152569` | 0.9677 | 171 | TREC-06 | legit | Reliable Computing, Vol.11, issue 3, 2005 │ Reliable Computing Volume 11, issue 3, 2005 Mathematical Research Enclosing Solutions of Singular Interval Systems I... |
+| 25 | `primary:011118` | 0.9674 | 143 | TREC-06 | legit |  │ Dear Sir, I am very mingled by the pine system. MY inbox massage are always eatten up without pior notification. All my in-coming mag disappear and nothing i... |
+
+## Three-way (legitimate / phishing / spam)
+
+No three-way ground truth on this split: no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth).
+
+### Predicted kind of the flagged rows (informational — no phishing-vs-spam ground truth on this split)
+
+| group | n | predicted Phishing | predicted Spam |
+|---|---:|---:|---:|
+| true positives flagged | 3,793 | 974 | 2,819 |
+| false positives flagged | 520 | 175 | 345 |
+| missed positives (kind the model would have given) | 101 | 35 | 66 |
+
+## Patterns (simple lexical statistics, not a model explanation)
+
+### Tokens over-represented in false negatives vs true positives (document frequency, smoothed ratio)
+
+| token | share of FN docs | share of TP docs | ratio |
+|---|---:|---:|---:|
+| `smailex` | 0.119 | 0.000 | 934.36 |
+| `blocki` | 0.119 | 0.000 | 934.36 |
+| `scanmail` | 0.119 | 0.000 | 934.36 |
+| `progra` | 0.119 | 0.000 | 934.36 |
+| `scanning` | 0.119 | 0.000 | 934.36 |
+| `matched` | 0.119 | 0.000 | 311.45 |
+| `tex-fonts` | 0.079 | 0.000 | 211.79 |
+| `pattern` | 0.129 | 0.001 | 201.82 |
+| `blocking` | 0.129 | 0.001 | 201.82 |
+| `tpn` | 0.069 | 0.001 | 112.12 |
+| `setting` | 0.129 | 0.001 | 91.74 |
+| `pif` | 0.079 | 0.001 | 70.60 |
+| `detected` | 0.119 | 0.002 | 62.29 |
+| `utah` | 0.119 | 0.002 | 62.29 |
+| `attachment` | 0.149 | 0.003 | 50.37 |
+| `math` | 0.129 | 0.003 | 48.05 |
+| `enroll` | 0.050 | 0.001 | 45.68 |
+| `seismic` | 0.050 | 0.001 | 45.68 |
+| `settings` | 0.129 | 0.003 | 37.37 |
+| `moved` | 0.119 | 0.003 | 37.37 |
+
+### Tokens over-represented in false positives vs true negatives
+
+| token | share of FP docs | share of TN docs | ratio |
+|---|---:|---:|---:|
+| `irc-list-web` | 0.054 | 0.000 | 1301.91 |
+| `quarantine` | 0.054 | 0.000 | 118.36 |
+| `foundstone` | 0.050 | 0.001 | 80.70 |
+| `skidmore` | 0.054 | 0.001 | 76.58 |
+| `monty` | 0.052 | 0.001 | 66.12 |
+| `bn3` | 0.060 | 0.002 | 27.15 |
+| `ips` | 0.054 | 0.002 | 26.57 |
+| `anyplace` | 0.060 | 0.002 | 26.16 |
+| `noc's` | 0.054 | 0.002 | 23.67 |
+| `https` | 0.108 | 0.006 | 19.12 |
+| `cgi` | 0.060 | 0.004 | 16.17 |
+| `procexp` | 0.054 | 0.003 | 15.69 |
+| `tcpview` | 0.054 | 0.003 | 15.69 |
+| `sysinternals` | 0.054 | 0.004 | 15.32 |
+| `filtered` | 0.052 | 0.003 | 15.14 |
+| `atkins` | 0.054 | 0.004 | 14.63 |
+| `rogue` | 0.056 | 0.004 | 14.49 |
+| `firewall` | 0.071 | 0.005 | 13.93 |
+| `inbox` | 0.058 | 0.004 | 13.79 |
+| `manage` | 0.079 | 0.006 | 13.64 |
+
+### Composition of the error groups (share of rows)
+
+| group | n | mailing-list / digest | HTML-heavy | very short (<20 content tokens) | truncated (>512) | lure phrase |
+|---|---:|---:|---:|---:|---:|---:|
+| FP | 520 | 19.6% | 0.0% | 2.3% | 29.0% | 1.2% |
+| TN | 11,888 | 29.7% | 0.2% | 0.4% | 23.8% | 1.3% |
+| FN | 101 | 8.9% | 0.0% | 0.0% | 12.9% | 1.0% |
+| TP | 3,793 | 2.8% | 0.1% | 0.2% | 15.3% | 6.4% |
+
+### Length confound (RUN_LOG entry 2)
+
+| group | n | mean tokens (full) | mean tokens fed | median (full) | % truncated |
+|---|---:|---:|---:|---:|---:|
+| TP | 3,793 | 283.9 | 232.2 | 207 | 15.3 |
+| TN | 11,888 | 827.0 | 294.4 | 269 | 23.8 |
+| FP | 520 | 1237.2 | 276.6 | 242 | 29.0 |
+| FN | 101 | 321.7 | 220.4 | 181 | 12.9 |
+| CORRECT | 15,681 | 695.6 | 279.4 | 247 | 21.8 |
+| WRONG | 621 | 1088.3 | 267.5 | 213 | 26.4 |
+| all legitimate | 12,408 | 844.2 | 293.7 | 268 | 24.0 |
+| all unwanted | 3,894 | 284.9 | 231.9 | 206 | 15.3 |
+| all label3=unwanted | 3,894 | 284.9 | 231.9 | 206 | 15.3 |
+
+| length bucket (full tokens) | n | positive rate | recall | FPR | accuracy |
+|---|---:|---:|---:|---:|---:|
+| <=64 | 1,363 | 0.431 | 0.9744 | 0.1070 | 0.9281 |
+| 65-128 | 2,538 | 0.290 | 0.9620 | 0.0394 | 0.9610 |
+| 129-256 | 4,594 | 0.260 | 0.9766 | 0.0341 | 0.9687 |
+| 257-512 | 4,232 | 0.184 | 0.9782 | 0.0287 | 0.9726 |
+| >512 | 3,575 | 0.166 | 0.9782 | 0.0507 | 0.9541 |
+
+## Summary
+
+- The classifier catches **97.41% of unwanted mail (phishing or spam)** at threshold 0.5 (101 of 3,894 missed) while flagging 4.19% of legitimate mail (520 false alarms). Recall is the headline because a missed phish is the costly error for a small-business inbox.
+- Holding false alarms to 1 in 100 legitimate emails keeps recall at 89.55% (threshold 0.866, flag when p >= threshold); at 1 in 1,000 it is 64.97%. The best-F1 threshold is 0.682 (F1 0.9397).
+- Three-way: this split has no phishing-vs-spam ground truth; of the 3,793 true positives the model flagged, it called 974 phishing and 2,819 spam (informational only).
+- On the 398-row credential-lure slice (37 documented phrases, 61.1% of its rows are labelled positive) recall is 99.59% with precision 97.58%: the highest-cost phish are caught at least as well as the average.
+- Length confound: missed unwanted mail averages 322 tokens vs 284 for caught; false positives average 1237 tokens vs 827 for correct legitimate mail (all legitimate 844, all unwanted 285).
+  Legitimate mail is ~3.0x longer than unwanted mail on average; the error groups do not follow the length gap, so length is not the main shortcut.
+- Recall by length is lowest in the 65-128-token bucket (96.2%, 736 positives).
+- Of the 520 false positives, 19.6% look like mailing-list or digest traffic, 0.0% are HTML-heavy and 2.3% are very short (TN reference: 29.7% / 0.2% / 0.4%).
+- Tokens most over-represented in missed unwanted mail vs caught: `smailex`, `blocki`, `scanmail`, `progra`, `scanning`, `matched`, `tex-fonts`, `pattern`, `blocking`, `tpn`.
+- Tokens most over-represented in false positives vs correct legitimate mail: `irc-list-web`, `quarantine`, `foundstone`, `skidmore`, `monty`, `bn3`, `ips`, `anyplace`, `noc's`, `https`.
diff --git a/presentation/reports/error_analysis_test3.md b/presentation/reports/error_analysis_test3.md
new file mode 100644
index 0000000000000000000000000000000000000000..48632f7d4369f6ec729588b75cddd5807d1ed884
--- /dev/null
+++ b/presentation/reports/error_analysis_test3.md
@@ -0,0 +1,244 @@
+# Error analysis — test3 split (three-way holdout), run `ft-focal` (2026-10-09 10:46)
+
+All quoted emails are rows of public corpora (source Assassin, CEAS-08, Enron, Nazario, Nigerian_Fraud, TREC-07); nothing here is private mail. Threshold 0.5. Binary positive class = Unwanted (phishing or spam), P(unwanted) = P(phishing) + P(spam).
+
+## Counts (binary roll-up)
+
+- rows 3,220 (positives 1,221, 37.9%); label3 counts legit 1,999, phish 521, spam 700
+- **recall (unwanted) 0.9836** — 20 missed unwanted emails (false negatives) out of 1,221
+- precision 0.9893 — 13 false alarms out of 1,214 flagged
+- F1 0.9864, accuracy 0.9898, specificity 0.9935
+- ROC-AUC 0.9993, PR-AUC 0.9989; recall at 1% FPR 0.9869, at 0.1% FPR 0.8788
+- credential-lure slice: 275 rows, 268 positive (97.5% label purity: slice precision cannot exceed what the phrase list selects), recall 1.0000, precision 1.0000, 0 missed
+
+## Top-25 false negatives (missed unwanted mail, most confident first)
+
+| # | doc_id | P(unwanted) | tokens (full) | source | label_set | preview (subject │ body, 160 chars) |
+|---:|---|---:|---:|---|---|---|
+| 1 | `primary:191966` | 0.0224 | 86 | Enron | spam | re : saturday . here new and │ good day : i tried to call your 3 three time but your phone is not available . i think you did a mistake during filling the form ... |
+| 2 | `primary:148887` | 0.0467 | 1097 | Enron | spam | southern california demographic analysis workshop announcement ( january charset = windows - 1252 " > │ analyzing southern california communities : demographic ... |
+| 3 | `aura:Nazario:000097` | 0.1782 | 39 | Nazario | phish | Oil Payment Approval │ Find the attached document via Google secure to view closing statement. All parties have completed Documents. please sign these at your e... |
+| 4 | `primary:010013` | 0.2112 | 135 | Enron | spam | new birthday calendar │ hi i ' m setting up a birthday calendar and need your help . just click the link below and enter your birthday details . ( it ' s quick ... |
+| 5 | `primary:007176` | 0.2300 | 281 | Assassin | spam | TOMORROW -- Password: Soleil -- TV Casting │ Tomorrow will be huge. Password: Soleil has become the premiere Thursday night in New York - with a chill after-wor... |
+| 6 | `aura:Nazario:000643` | 0.2584 | 368 | Nazario | phish | Phishing Email Corpus │ Good Morning, Jose, My name is Michael Kotson, and I am a Cybersecurity researcher at MIT Lincoln Laboratory. I am developing a lecture ... |
+| 7 | `primary:181246` | 0.2858 | 1051 | Assassin | spam | [ILUG] (no subject) │ Suppliers of Computers, Printers, etc. & Consumables Central Data Supplies 24, Middleton Road, Banbury, Oxon, OX16 4QJ. Tel: 01295 271666 ... |
+| 8 | `primary:095405` | 0.2967 | 40 | Enron | spam | pocket micro scope id : sxo 683 │ re : ppql 308243023 not interested , please let us know : powerlinepromo @ yahoo . com |
+| 9 | `primary:014300` | 0.3203 | 301 | Enron | spam | cecil county networking lunch │ networking lunch chesapeake bay golf club in north east wednesday , november 17 th 11 : 15 am smallcompanies . com in conjunctio... |
+| 10 | `primary:001939` | 0.3348 | 1153 | Assassin | spam | Invite: Content Management Summit, Oct. 10th New York City │ ![CMS](http://www.cmsummit.com/images/right_02.gif) ![CMS](http://www.cmsummit.com/images/cms_title... |
+| 11 | `primary:033220` | 0.3678 | 28444 | Assassin | spam | Wir suchen Erotikdarsteller! │ This is a Multipart MIME message. Since your mail reader does not understand this format, some or all of this message may not be ... |
+| 12 | `primary:025244` | 0.3789 | 8 | Enron | spam | message subject │ goodbye |
+| 13 | `primary:133074` | 0.3791 | 86 | Enron | spam | his lips , his nose , │ good day : i tried to call your 3 three time but your phone is not available . i think you did a mistake during filling the form . anywa... |
+| 14 | `aura:Nigerian_Fraud:003049` | 0.3823 | 516 | Nigerian_Fraud | phish | Invitation from aditya monu │ This is a multi-part message in MIME format. --incrowd_6506e2e6636163656626f7892e636f26d05b3743 Content-Type: text/plain; charset=... |
+| 15 | `aura:Nazario:000299` | 0.3973 | 46 | Nazario | phish | RE: IT Administrator │ IT Administrator desk: has currently upgraded our mailbox service quote to 150GB space Faculty/Staff are hereby advice to click on WEBMAI... |
+| 16 | `primary:046569` | 0.4039 | 1078 | Enron | spam | invite : content management summit , oct . 10 th new york city │ dear friends , the online content business is doing fantastic . sound crazy ? perhaps . with ad... |
+| 17 | `primary:135631` | 0.4338 | 18 | Enron | spam | reply : shy mom wants to date you │ hello there ! sode |
+| 18 | `primary:148857` | 0.4750 | 180 | Assassin | spam | send REAL PAPER GREETING CARDS on-line! │ Cards In Advance Birthday Anniversary Get Well Soon Thinking of you Congratulations Sympathy Engagement Good Luck New ... |
+| 19 | `primary:129677` | 0.4929 | 510 | Assassin | spam | Proposal for cooperation │ Hello. I meant to contact http://taint.org webmaster, Am I correct, I hope? Actually, I am not quite sure... Did I or Natasha Fedorov... |
+| 20 | `primary:075760` | 0.4969 | 719 | Enron | spam | greece based professionals and organizations │ you areinvited to register on training consortiumjoin us by registering for free as a trainer , consultant , or s... |
+
+## Top-25 false positives (legitimate flagged, most confident first)
+
+| # | doc_id | P(unwanted) | tokens (full) | source | label_set | preview (subject │ body, 160 chars) |
+|---:|---|---:|---:|---|---|---|
+| 1 | `primary:144048` | 0.9853 | 46 | CEAS-08 | legit | Whiskas Scratching Post │ HTML Message - =?windows-1252?B?V2hpc2thcyBTY3JhdGNoaW5nIFBvc3QA?= |
+| 2 | `primary:199386` | 0.9471 | 2206 | Assassin | legit | Back to School with Buy4Now │ This is a multi-part message in MIME format. ------=_NextPart_000_22A2C_01C23F09.09760420 Content-Type: text/plain; charset="iso-8... |
+| 3 | `primary:091801` | 0.9436 | 66 | CEAS-08 | legit | Lots Of SPAM │ Hi List, i have posted my RAW email in http://pastebin.ca/918849 , i'm receiving 1000 to 4000 per day this king of mesages. SA also skipping this... |
+| 4 | `primary:152763` | 0.8427 | 1228 | CEAS-08 | legit | SAVE 30% on Best Recipe Cookbooks │ Untitled Document We don't want you to miss out on our e-mail offers! Please add our e-mail address, arfldpykmjrzc@americast... |
+| 5 | `primary:165228` | 0.8246 | 4917 | Assassin | legit | United We Fall? │ This is a multi-part message in MIME format... ------------=_1029244856-11739-4 Content-Type: text/plain; charset="iso-8859-1" Content-Disposi... |
+| 6 | `primary:041713` | 0.7668 | 340 | TREC-07 | legit | Electric Transmission Week complimentary trial │ 1105_ETW_Trial View this message as a Web page. FREE Trial Offer You're invited to receive a FREE trial subscri... |
+| 7 | `primary:103927` | 0.7592 | 1587 | CEAS-08 | legit | FREE Shipping on "America's Best Lost Recipes" Cookbook - Plus Free OXO(R) Tongs │ Untitled Document We don't want you to miss out on our e-mail offers! Please ... |
+| 8 | `primary:055965` | 0.7247 | 5197 | Assassin | legit | Brother, Can You Spare a Jet? │ This is a multi-part message in MIME format... ------------=_1029157891-23956-4 Content-Type: text/plain; charset="iso-8859-1" C... |
+| 9 | `primary:170237` | 0.6660 | 88 | CEAS-08 | legit | The jet set │ Well, it appears you have all taken an idea and run with it. My point was that "Knocked Up" appeared quite the good film, and the Williams one so ... |
+| 10 | `primary:117398` | 0.6510 | 5959 | Assassin | legit | AOL Keyword: Fraud Probe │ This is a multi-part message in MIME format... ------------=_1033046883-23957-4 Content-Type: text/plain; charset="iso-8859-1" Conten... |
+| 11 | `primary:168120` | 0.5796 | 268 | TREC-07 | legit | Please confirm your request to join Deficiv │ Hello deficient@speedy.uwaterloo.ca, We have received your request to join the Deficiv group hosted by Yahoo! Grou... |
+| 12 | `primary:057514` | 0.5173 | 809 | CEAS-08 | legit | WIN a $6,000 Cook Islands getaway │ HotelClub Newsletter Dear Tony, Subscribe to any free NEWS.com.au newsletter for your chance to WIN a $6,000 Cook Islands ge... |
+| 13 | `primary:021911` | 0.5051 | 875 | CEAS-08 | legit | It's here. Dell's first tablet PC. │ Dell would like to introduce our first tablet PC, the Latitude XT. Part of the trusted Latitude line, the XT gives you all ... |
+
+## Three-way (legitimate / phishing / spam)
+
+Decision rule: unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.
+
+Rows with a full label: 3,220; unwanted-but-unsplit rows (binary roll-up only): 0.
+
+| true \ predicted | Legitimate | Phishing | Spam |
+|---|---:|---:|---:|
+| **Legitimate** | 1,986 | 1 | 12 |
+| **Phishing** | 4 | 514 | 3 |
+| **Spam** | 16 | 1 | 683 |
+
+- accuracy 0.9885, **macro-F1 0.9867** (over 3 classes; over the 3 present: 0.9867); argmax-of-3 rule: macro-F1 0.9858, 6 rows differ between the rules
+
+| class | precision | recall | F1 | support |
+|---|---:|---:|---:|---:|
+| Legitimate | 0.9900 | 0.9935 | 0.9918 | 1,999 |
+| Phishing | 0.9961 | 0.9866 | 0.9913 | 521 |
+| Spam | 0.9785 | 0.9757 | 0.9771 | 700 |
+
+### Phishing vs spam on the 1,221 true unwanted rows
+
+Kind decision alone (given the row is unwanted, argmax(P(phishing), P(spam))): accuracy 0.9959; phishing read as spam 4, spam read as phishing 1.
+
+| true \ predicted (roll-up rule) | Legitimate (missed) | Phishing | Spam |
+|---|---:|---:|---:|
+| **Phishing** | 4 | 514 | 3 |
+| **Spam** | 16 | 1 | 683 |
+
+### Per source (three-way rows)
+
+| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |
+|---|---:|---:|---:|---:|---:|---:|
+| Assassin | 236 | 164 | 0 | 72 | 0.9534 | 0.9458 |
+| CEAS-08 | 736 | 736 | 0 | 0 | 0.9905 | 0.9952 |
+| Enron | 628 | 0 | 0 | 628 | 0.9841 | 0.9920 |
+| Nazario | 222 | 0 | 222 | 0 | 0.9865 | 0.9932 |
+| Nigerian_Fraud | 299 | 0 | 299 | 0 | 0.9866 | 0.9933 |
+| TREC-07 | 1,099 | 1,099 | 0 | 0 | 0.9982 | 0.9991 |
+
+### Top-25 phishing -> spam confusions (3 total, most confident first)
+
+| # | doc_id | P(legit) | P(phish) | P(spam) | tokens (full) | source | preview (subject │ body, 160 chars) |
+|---:|---|---:|---:|---:|---:|---|---|
+| 1 | `aura:Nigerian_Fraud:001698` | 0.0949 | 0.0320 | 0.8730 | 2061 | Nigerian_Fraud | " Please add me to your mailing list. " │ "How to turn 3 into 10,000" This is not a get rich quick scheme, but with a little effort will make a nice little retu... |
+| 2 | `aura:Nigerian_Fraud:000337` | 0.1620 | 0.3029 | 0.5352 | 330 | Nigerian_Fraud | Dddddddd R. Rrrrr Message from Vasil Nanchev. │ Hi, This is Vasil Nanchev. I just wanted to inform you about something which is really time sensitive. For the f... |
+| 3 | `aura:Nigerian_Fraud:000824` | 0.1349 | 0.3997 | 0.4654 | 1089 | Nigerian_Fraud | Seeking joint-venture partners for ERW oil casing project │ Dear Sir or Madam: We are keenly interested in setting up a new factory to produce 120,000 ERW oil c... |
+
+### Top-25 spam -> phishing confusions (1 total, most confident first)
+
+| # | doc_id | P(legit) | P(phish) | P(spam) | tokens (full) | source | preview (subject │ body, 160 chars) |
+|---:|---|---:|---:|---:|---:|---|---|
+| 1 | `primary:077835` | 0.0080 | 0.8669 | 0.1251 | 1179 | Assassin | URGENT BUSINESS ASSISTANCE. │ --0-1118764524-1030968705=:44128 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit KEVIN GEORGE ABIDJAN... |
+
+### Tokens over-represented in phishing read as spam vs phishing read correctly
+
+| token | share of phish->spam docs | share of phish->phish docs | ratio |
+|---|---:|---:|---:|
+| `join` | 1.000 | 0.019 | 49.00 |
+| `could` | 1.000 | 0.076 | 13.03 |
+| `www` | 1.000 | 0.150 | 6.64 |
+| `new` | 1.000 | 0.160 | 6.24 |
+| `http` | 1.000 | 0.193 | 5.17 |
+| `years` | 1.000 | 0.196 | 5.07 |
+| `get` | 1.000 | 0.202 | 4.92 |
+| `about` | 1.000 | 0.239 | 4.17 |
+| `after` | 1.000 | 0.278 | 3.59 |
+| `who` | 1.000 | 0.317 | 3.15 |
+| `into` | 1.000 | 0.323 | 3.09 |
+| `com` | 1.000 | 0.339 | 2.95 |
+| `which` | 1.000 | 0.362 | 2.76 |
+| `now` | 1.000 | 0.379 | 2.63 |
+| `can` | 1.000 | 0.467 | 2.14 |
+| `are` | 1.000 | 0.562 | 1.78 |
+| `from` | 1.000 | 0.644 | 1.55 |
+| `will` | 1.000 | 0.714 | 1.40 |
+| `for` | 1.000 | 0.800 | 1.25 |
+| `this` | 1.000 | 0.815 | 1.23 |
+
+### Tokens over-represented in spam read as phishing vs spam read correctly
+
+(too few rows in one of the groups)
+
+### Predicted kind of the flagged rows (informational — includes the unsplit rows, which have no kind label)
+
+| group | n | predicted Phishing | predicted Spam |
+|---|---:|---:|---:|
+| true positives flagged | 1,201 | 515 | 686 |
+| false positives flagged | 13 | 1 | 12 |
+| missed positives (kind the model would have given) | 20 | 3 | 17 |
+
+## Patterns (simple lexical statistics, not a model explanation)
+
+### Tokens over-represented in false negatives vs true positives (document frequency, smoothed ratio)
+
+| token | share of FN docs | share of TP docs | ratio |
+|---|---:|---:|---:|
+| `invite` | 0.250 | 0.002 | 128.94 |
+| `cut` | 0.150 | 0.002 | 82.05 |
+| `practical` | 0.150 | 0.002 | 82.05 |
+| `networking` | 0.200 | 0.002 | 75.36 |
+| `audience` | 0.150 | 0.003 | 45.59 |
+| `oct` | 0.150 | 0.003 | 45.59 |
+| `innovative` | 0.150 | 0.003 | 45.59 |
+| `event` | 0.200 | 0.005 | 40.58 |
+| `angeles` | 0.150 | 0.004 | 37.30 |
+| `anyway` | 0.150 | 0.004 | 37.30 |
+| `feature` | 0.150 | 0.005 | 31.56 |
+| `york` | 0.150 | 0.005 | 31.56 |
+| `subscription` | 0.200 | 0.007 | 27.76 |
+| `technologies` | 0.150 | 0.006 | 27.35 |
+| `entertainment` | 0.150 | 0.007 | 24.13 |
+| `staff` | 0.200 | 0.011 | 19.54 |
+| `association` | 0.150 | 0.008 | 19.54 |
+| `markets` | 0.150 | 0.009 | 17.84 |
+| `hosting` | 0.150 | 0.009 | 17.84 |
+| `mailman` | 0.150 | 0.009 | 17.84 |
+
+### Tokens over-represented in false positives vs true negatives
+
+| token | share of FP docs | share of TN docs | ratio |
+|---|---:|---:|---:|
+| `outlets` | 0.231 | 0.001 | 206.01 |
+| `zzz-unspun` | 0.231 | 0.001 | 206.01 |
+| `czzz-unspun` | 0.231 | 0.001 | 206.01 |
+| `cook` | 0.231 | 0.001 | 206.01 |
+| `full-color` | 0.385 | 0.002 | 179.85 |
+| `muehlbauer` | 0.231 | 0.002 | 147.15 |
+| `vanderkolk` | 0.231 | 0.002 | 147.15 |
+| `eletra` | 0.231 | 0.002 | 147.15 |
+| `loripatel` | 0.231 | 0.002 | 147.15 |
+| `copyedited` | 0.231 | 0.002 | 147.15 |
+| `jimduffy86` | 0.231 | 0.002 | 147.15 |
+| `asbrand` | 0.231 | 0.002 | 147.15 |
+| `patel` | 0.231 | 0.002 | 147.15 |
+| `authoritatively` | 0.231 | 0.002 | 147.15 |
+| `four-week` | 0.231 | 0.002 | 147.15 |
+| `duffy` | 0.231 | 0.002 | 147.15 |
+| `mediaunspun` | 0.231 | 0.002 | 147.15 |
+| `savviest` | 0.231 | 0.002 | 147.15 |
+| `dasbrand` | 0.231 | 0.002 | 147.15 |
+| `sendashtml` | 0.231 | 0.002 | 147.15 |
+
+### Composition of the error groups (share of rows)
+
+| group | n | mailing-list / digest | HTML-heavy | very short (<20 content tokens) | truncated (>512) | lure phrase |
+|---|---:|---:|---:|---:|---:|---:|
+| FP | 13 | 46.2% | 0.0% | 0.0% | 61.5% | 0.0% |
+| TN | 1,986 | 56.3% | 0.0% | 0.0% | 38.3% | 0.4% |
+| FN | 20 | 30.0% | 0.0% | 10.0% | 35.0% | 0.0% |
+| TP | 1,201 | 8.3% | 0.0% | 0.6% | 26.6% | 22.3% |
+
+### Length confound (RUN_LOG entry 2)
+
+| group | n | mean tokens (full) | mean tokens fed | median (full) | % truncated |
+|---|---:|---:|---:|---:|---:|
+| TP | 1,201 | 978.0 | 279.8 | 243 | 26.6 |
+| TN | 1,986 | 732.6 | 358.4 | 392 | 38.3 |
+| FP | 13 | 1814.3 | 377.2 | 875 | 61.5 |
+| FN | 20 | 1807.8 | 284.1 | 291 | 35.0 |
+| CORRECT | 3,187 | 825.1 | 328.8 | 339 | 33.9 |
+| WRONG | 33 | 1810.4 | 320.8 | 368 | 45.5 |
+| all legitimate | 1,999 | 739.7 | 358.6 | 393 | 38.5 |
+| all unwanted | 1,221 | 991.6 | 279.9 | 243 | 26.7 |
+| all label3=phish | 521 | 1349.4 | 355.1 | 412 | 39.9 |
+| all label3=spam | 700 | 725.3 | 223.9 | 159 | 16.9 |
+
+| length bucket (full tokens) | n | positive rate | recall | FPR | accuracy |
+|---|---:|---:|---:|---:|---:|
+| <=64 | 129 | 0.767 | 0.9495 | 0.0333 | 0.9535 |
+| 65-128 | 427 | 0.590 | 0.9921 | 0.0114 | 0.9906 |
+| 129-256 | 699 | 0.395 | 0.9928 | 0.0000 | 0.9971 |
+| 257-512 | 870 | 0.308 | 0.9851 | 0.0033 | 0.9931 |
+| >512 | 1,095 | 0.298 | 0.9785 | 0.0104 | 0.9863 |
+
+## Summary
+
+- The classifier catches **98.36% of unwanted mail (phishing or spam)** at threshold 0.5 (20 of 1,221 missed) while flagging 0.65% of legitimate mail (13 false alarms). Recall is the headline because a missed phish is the costly error for a small-business inbox.
+- Holding false alarms to 1 in 100 legitimate emails keeps recall at 98.69% (threshold 0.434, flag when p >= threshold); at 1 in 1,000 it is 87.88%. The best-F1 threshold is 0.475 (F1 0.9877).
+- Three-way: macro-F1 **0.9867** over 3,220 rows with a full label (recall legitimate 99.3%, phishing 98.7%, spam 97.6%). Given that an email is unwanted, the model tells phishing from spam 99.6% of the time (4 phishing read as spam, 1 spam read as phishing). The phishing/spam labels follow corpus provenance (Nazario + Nigerian fraud + synthetic lures = phishing; SpamAssassin + Enron-Spam spam folders = spam), so part of this confusion is corpus style, not lure content.
+- On the 275-row credential-lure slice (37 documented phrases, 97.5% of its rows are labelled positive) recall is 100.00% with precision 100.00%: the highest-cost phish are caught at least as well as the average.
+- Length confound: missed unwanted mail averages 1808 tokens vs 978 for caught; false positives average 1814 tokens vs 733 for correct legitimate mail (all legitimate 740, all unwanted 992).
+  Legitimate mail is ~0.7x longer than unwanted mail on average; the error groups do not follow the length gap, so length is not the main shortcut.
+  Phishing rows average 1349 tokens vs spam 725: a length gap between the two unwanted kinds is a second shortcut to watch in the phishing-vs-spam confusion.
+- Recall by length is lowest in the <=64-token bucket (94.9%, 99 positives).
+- Of the 13 false positives, 46.2% look like mailing-list or digest traffic, 0.0% are HTML-heavy and 0.0% are very short (TN reference: 56.3% / 0.0% / 0.0%).
+- Tokens most over-represented in missed unwanted mail vs caught: `invite`, `cut`, `practical`, `networking`, `audience`, `oct`, `innovative`, `event`, `angeles`, `anyway`.
+- Tokens most over-represented in false positives vs correct legitimate mail: `outlets`, `zzz-unspun`, `czzz-unspun`, `cook`, `full-color`, `muehlbauer`, `vanderkolk`, `eletra`, `loripatel`, `copyedited`.
diff --git a/presentation/reports/eval_finetune_curves.png b/presentation/reports/eval_finetune_curves.png
new file mode 100644
index 0000000000000000000000000000000000000000..c696075eb769f34e0fcfc21c964efafd55565ab3
--- /dev/null
+++ b/presentation/reports/eval_finetune_curves.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:791d920eac59297235d6dc21c232d689b2286b045264a54f02fe0ec45cd7107e
+size 140610
diff --git a/presentation/reports/eval_finetune_curves_test3.png b/presentation/reports/eval_finetune_curves_test3.png
new file mode 100644
index 0000000000000000000000000000000000000000..c696075eb769f34e0fcfc21c964efafd55565ab3
--- /dev/null
+++ b/presentation/reports/eval_finetune_curves_test3.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:791d920eac59297235d6dc21c232d689b2286b045264a54f02fe0ec45cd7107e
+size 140610
diff --git a/presentation/reports/eval_ft-12L/metrics.json b/presentation/reports/eval_ft-12L/metrics.json
new file mode 100644
index 0000000000000000000000000000000000000000..a665078a649f7940d2c4350aa9e929dfb4c1fb5e
--- /dev/null
+++ b/presentation/reports/eval_ft-12L/metrics.json
@@ -0,0 +1,1109 @@
+{
+  "timestamp": "2026-10-09 08:54",
+  "run_name": "ft-12L",
+  "split": "test",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet",
+  "split_title": "TREC-06 binary holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-12L/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 12,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 186290179,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-12L",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 12,
+      "freeze_layers": 0,
+      "loss": "cross_entropy",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 1,
+    "best_step": 6250,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9925057415689592,
+    "val_metrics": {
+      "accuracy": 0.9918869405914682,
+      "precision": 0.9935866408518877,
+      "recall": 0.9914271914996378,
+      "f1": 0.9925057415689592,
+      "roc_auc": 0.9995007118744352,
+      "pr_auc": 0.9995737781814353,
+      "tn": 6949,
+      "fp": 53,
+      "fn": 71,
+      "tp": 8211,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.17105858027935028,
+      "f1_at_optimal_threshold": 0.9928829915560917,
+      "f1_unwanted": 0.9925057415689592,
+      "n_3way": 8105,
+      "accuracy_3way": 0.9916101172115978,
+      "macro_f1_3way": 0.9774079655816474,
+      "precision_legitimate": 0.9984195402298851,
+      "recall_legitimate": 0.9924307340759783,
+      "f1_legitimate": 0.9954161294943418,
+      "precision_phishing": 0.9570200573065902,
+      "recall_phishing": 0.9823529411764705,
+      "f1_phishing": 0.969521044992743,
+      "precision_spam": 0.9472361809045227,
+      "recall_spam": 0.9882044560943644,
+      "f1_spam": 0.9672867222578576,
+      "cm3_ll": 6949,
+      "cm3_lp": 13,
+      "cm3_ls": 40,
+      "cm3_pl": 4,
+      "cm3_pp": 334,
+      "cm3_ps": 2,
+      "cm3_sl": 7,
+      "cm3_sp": 2,
+      "cm3_ss": 754,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 338,
+      "pvs_phish_as_spam": 2,
+      "pvs_spam_as_phish": 2,
+      "pvs_spam_as_spam": 761,
+      "val_loss": 0.035248381216093624,
+      "n": 15284,
+      "eval_seconds": 22.8
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9918869405914682,
+        "precision": 0.9935866408518877,
+        "recall": 0.9914271914996378,
+        "f1": 0.9925057415689592,
+        "roc_auc": 0.9995007118744352,
+        "pr_auc": 0.9995737781814353,
+        "tn": 6949,
+        "fp": 53,
+        "fn": 71,
+        "tp": 8211,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.17105858027935028,
+        "f1_at_optimal_threshold": 0.9928829915560917,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.9916101172115978,
+        "macro_f1": 0.9774079655816474,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9984195402298851,
+            "recall": 0.9924307340759783,
+            "f1": 0.9954161294943418
+          },
+          "Phishing": {
+            "precision": 0.9570200573065902,
+            "recall": 0.9823529411764705,
+            "f1": 0.969521044992743
+          },
+          "Spam": {
+            "precision": 0.9472361809045227,
+            "recall": 0.9882044560943644,
+            "f1": 0.9672867222578576
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6949,
+              13,
+              40
+            ],
+            [
+              4,
+              334,
+              2
+            ],
+            [
+              7,
+              2,
+              754
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              338,
+              2
+            ],
+            [
+              2,
+              761
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.17105858027935028,
+      "val_f1_at_optimal": 0.9928829915560917
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 186290179,
+      "trainable": 186290179,
+      "embeddings": 34080768,
+      "encoder_layers": 151154688,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 16302,
+    "positive": 3894,
+    "negative": 12408,
+    "positive_rate": 0.2389,
+    "label3": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "by_source": {
+      "TREC-06": {
+        "rows": 16302,
+        "positive": 3894,
+        "label3": {
+          "legit": 12408,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 3894
+        }
+      }
+    },
+    "mean_tokens_full": 710.6,
+    "pct_truncated": 21.93
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9347932768985401,
+    "precision": 0.8119903019616487,
+    "recall": 0.9460708782742681,
+    "f1": 0.8739176847349069,
+    "specificity": 0.931254029658285,
+    "false_positive_rate": 0.06874597034171502,
+    "false_negative_rate": 0.053929121725731895,
+    "confusion_matrix": {
+      "tn": 11555,
+      "fp": 853,
+      "fn": 210,
+      "tp": 3684
+    },
+    "n": 16302,
+    "n_positive": 3894,
+    "n_negative": 12408
+  },
+  "threshold_free": {
+    "roc_auc": 0.9835520193907075,
+    "pr_auc": 0.9512527318693685,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.6607601438109912,
+      "fpr": 0.009993552546744036,
+      "threshold": 0.9997559785842896,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.40729327170005136,
+      "fpr": 0.0009671179883945841,
+      "threshold": 0.999977171421051,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.9551892876625061,
+      "f1": 0.8851884312007011,
+      "precision": 0.8636696799413633,
+      "recall": 0.9078068823831535
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 3684,
+        "predicted_phishing": 802,
+        "predicted_spam": 2882
+      },
+      "false_positives_flagged": {
+        "n": 853,
+        "predicted_phishing": 366,
+        "predicted_spam": 487
+      },
+      "missed_positives": {
+        "n": 210,
+        "predicted_phishing": 13,
+        "predicted_spam": 197
+      },
+      "all_flagged": {
+        "n": 4537,
+        "predicted_phishing": 1168,
+        "predicted_spam": 3369
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 3684,
+        "predicted_phishing": 802,
+        "predicted_spam": 2882
+      }
+    },
+    "rows_with_full_label": 12408,
+    "rows_unwanted_only": 3894,
+    "evaluated": false,
+    "reason": "no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)"
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 129,
+      "bank account": 78,
+      "security alert": 70,
+      "within 24 hours": 65,
+      "urgently": 51,
+      "paypal account": 20,
+      "within 48 hours": 7,
+      "verify your account": 7,
+      "update your account": 6,
+      "account will be suspended": 5,
+      "ebay account": 5,
+      "enter your password": 5,
+      "wire transfer": 5,
+      "unusual activity": 5,
+      "confirm your identity": 4,
+      "verify your identity": 4,
+      "click here to verify": 3,
+      "log in to your account": 2,
+      "confirm your account": 2,
+      "security notice": 1,
+      "gift card": 1,
+      "reset your password": 1,
+      "account has been locked": 1,
+      "update your information": 1,
+      "account has been limited": 1,
+      "login to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 5,
+        "negative": 0
+      },
+      "update your account": {
+        "positive": 6,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 0,
+        "negative": 1
+      },
+      "click here to verify": {
+        "positive": 3,
+        "negative": 0
+      },
+      "account has been limited": {
+        "positive": 1,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 4,
+        "negative": 1
+      },
+      "account has been locked": {
+        "positive": 1,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 2,
+        "negative": 3
+      },
+      "paypal account": {
+        "positive": 20,
+        "negative": 0
+      },
+      "ebay account": {
+        "positive": 5,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "wire transfer": {
+        "positive": 5,
+        "negative": 0
+      },
+      "gift card": {
+        "positive": 0,
+        "negative": 1
+      },
+      "urgent": {
+        "positive": 81,
+        "negative": 48
+      },
+      "urgently": {
+        "positive": 22,
+        "negative": 29
+      },
+      "security alert": {
+        "positive": 2,
+        "negative": 68
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "login to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 71,
+        "negative": 7
+      },
+      "within 24 hours": {
+        "positive": 63,
+        "negative": 2
+      },
+      "within 48 hours": {
+        "positive": 4,
+        "negative": 3
+      }
+    },
+    "label_purity": 0.6106,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 155,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 243
+    },
+    "threeway_accuracy": null,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 0.9321608040201005,
+      "precision": 0.9029850746268657,
+      "recall": 0.9958847736625515,
+      "f1": 0.9471624266144814,
+      "specificity": 0.832258064516129,
+      "false_positive_rate": 0.16774193548387098,
+      "false_negative_rate": 0.00411522633744856,
+      "confusion_matrix": {
+        "tn": 129,
+        "fp": 26,
+        "fn": 1,
+        "tp": 242
+      },
+      "n": 398,
+      "n_positive": 243,
+      "n_negative": 155,
+      "roc_auc": 0.9919288464091331
+    }
+  },
+  "per_source": {
+    "TREC-06": {
+      "threshold": 0.5,
+      "accuracy": 0.9347932768985401,
+      "precision": 0.8119903019616487,
+      "recall": 0.9460708782742681,
+      "f1": 0.8739176847349069,
+      "specificity": 0.931254029658285,
+      "false_positive_rate": 0.06874597034171502,
+      "false_negative_rate": 0.053929121725731895,
+      "confusion_matrix": {
+        "tn": 11555,
+        "fp": 853,
+        "fn": 210,
+        "tp": 3684
+      },
+      "n": 16302,
+      "n_positive": 3894,
+      "n_negative": 12408,
+      "roc_auc": 0.9835520193907075,
+      "label3_counts": {
+        "legit": 12408,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 3894
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 3684,
+        "mean_tokens_full": 286.3,
+        "mean_tokens_fed": 233.1,
+        "median_tokens_full": 208.0,
+        "pct_truncated": 15.1
+      },
+      "tn": {
+        "n": 11555,
+        "mean_tokens_full": 824.9,
+        "mean_tokens_fed": 294.1,
+        "median_tokens_full": 269.0,
+        "pct_truncated": 23.5
+      },
+      "fp": {
+        "n": 853,
+        "mean_tokens_full": 1105.4,
+        "mean_tokens_fed": 288.0,
+        "median_tokens_full": 249.0,
+        "pct_truncated": 30.9
+      },
+      "fn": {
+        "n": 210,
+        "mean_tokens_full": 260.3,
+        "mean_tokens_fed": 210.7,
+        "median_tokens_full": 134.0,
+        "pct_truncated": 18.6
+      },
+      "correct": {
+        "n": 15239,
+        "mean_tokens_full": 694.7,
+        "mean_tokens_fed": 279.3,
+        "median_tokens_full": 247.0,
+        "pct_truncated": 21.5
+      },
+      "wrong": {
+        "n": 1063,
+        "mean_tokens_full": 938.4,
+        "mean_tokens_fed": 272.7,
+        "median_tokens_full": 221.0,
+        "pct_truncated": 28.5
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 1363,
+        "n_positive": 587,
+        "positive_rate": 0.43066764490095377,
+        "recall": 0.9063032367972743,
+        "fpr": 0.13015463917525774,
+        "accuracy": 0.8855465884079237
+      },
+      {
+        "bucket": "65-128",
+        "n": 2538,
+        "n_positive": 736,
+        "positive_rate": 0.2899921197793538,
+        "recall": 0.9320652173913043,
+        "fpr": 0.07935627081021088,
+        "accuracy": 0.9239558707643813
+      },
+      {
+        "bucket": "129-256",
+        "n": 4594,
+        "n_positive": 1196,
+        "positive_rate": 0.260339573356552,
+        "recall": 0.9690635451505016,
+        "fpr": 0.05503237198351972,
+        "accuracy": 0.9512407488027862
+      },
+      {
+        "bucket": "257-512",
+        "n": 4232,
+        "n_positive": 780,
+        "positive_rate": 0.1843100189035917,
+        "recall": 0.9628205128205128,
+        "fpr": 0.04577056778679027,
+        "accuracy": 0.9558128544423441
+      },
+      {
+        "bucket": ">512",
+        "n": 3575,
+        "n_positive": 595,
+        "positive_rate": 0.16643356643356644,
+        "recall": 0.934453781512605,
+        "fpr": 0.08859060402684564,
+        "accuracy": 0.9152447552447552
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 16302,
+    "seconds": 27.55,
+    "rows_per_s": 591.7,
+    "padded_tokens": 4606544,
+    "real_tokens": 4546874,
+    "padded_tokens_per_s": 167190,
+    "peak_vram_gb": 1.7
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 5.05,
+      "p50_ms": 4.42,
+      "p95_ms": 6.98,
+      "max_ms": 12.33,
+      "avg_s": 0.0051,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.068,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 246.62,
+      "p50_ms": 242.67,
+      "p95_ms": 356.11,
+      "max_ms": 389.87,
+      "avg_s": 0.2466,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 3.333,
+      "threads": 16,
+      "estimated_full_run_s": 27.1
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-12L/best",
+    "run_name": "ft-12L",
+    "params": 186290179,
+    "timestamp": "2026-10-09 08:54",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-12L/error_analysis.md",
+    "n_fn": 210,
+    "n_fp": 853,
+    "composition": {
+      "fp": {
+        "n": 853,
+        "mailing_list_or_digest": 0.164,
+        "html_heavy": 0.002,
+        "very_short_lt20_tokens": 0.011,
+        "truncated_gt512": 0.309,
+        "lure_phrase": 0.03
+      },
+      "tn": {
+        "n": 11555,
+        "mailing_list_or_digest": 0.302,
+        "html_heavy": 0.002,
+        "very_short_lt20_tokens": 0.005,
+        "truncated_gt512": 0.235,
+        "lure_phrase": 0.011
+      },
+      "fn": {
+        "n": 210,
+        "mailing_list_or_digest": 0.052,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.01,
+        "truncated_gt512": 0.186,
+        "lure_phrase": 0.005
+      },
+      "tp": {
+        "n": 3684,
+        "mailing_list_or_digest": 0.029,
+        "html_heavy": 0.001,
+        "very_short_lt20_tokens": 0.002,
+        "truncated_gt512": 0.151,
+        "lure_phrase": 0.066
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "yhoo",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "greko's",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "pre-registration",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "gannett",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "e-marketing",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "inundated",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "usages",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "producing",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "nasdaq",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "goog",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "ibd",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "ringing",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "gci",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "twx",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "fotis",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "targeting",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 297.56
+      },
+      {
+        "token": "dgko",
+        "share_a": 0.119,
+        "share_b": 0.001,
+        "ratio": 178.54
+      },
+      {
+        "token": "dow",
+        "share_a": 0.119,
+        "share_b": 0.001,
+        "ratio": 178.54
+      },
+      {
+        "token": "clixme",
+        "share_a": 0.119,
+        "share_b": 0.001,
+        "ratio": 178.54
+      },
+      {
+        "token": "greko",
+        "share_a": 0.119,
+        "share_b": 0.001,
+        "ratio": 178.54
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "forwarding",
+        "share_a": 0.084,
+        "share_b": 0.014,
+        "ratio": 5.97
+      },
+      {
+        "token": "esmtp",
+        "share_a": 0.08,
+        "share_b": 0.014,
+        "ratio": 5.85
+      },
+      {
+        "token": "hotmail",
+        "share_a": 0.064,
+        "share_b": 0.011,
+        "ratio": 5.67
+      },
+      {
+        "token": "inform",
+        "share_a": 0.049,
+        "share_b": 0.009,
+        "ratio": 5.56
+      },
+      {
+        "token": "precedence",
+        "share_a": 0.053,
+        "share_b": 0.01,
+        "ratio": 5.48
+      },
+      {
+        "token": "coordinator",
+        "share_a": 0.054,
+        "share_b": 0.01,
+        "ratio": 5.31
+      },
+      {
+        "token": "content-transfer-encoding",
+        "share_a": 0.07,
+        "share_b": 0.014,
+        "ratio": 4.95
+      },
+      {
+        "token": "mime-version",
+        "share_a": 0.086,
+        "share_b": 0.017,
+        "ratio": 4.91
+      },
+      {
+        "token": "safe",
+        "share_a": 0.054,
+        "share_b": 0.012,
+        "ratio": 4.65
+      },
+      {
+        "token": "private",
+        "share_a": 0.069,
+        "share_b": 0.016,
+        "ratio": 4.46
+      },
+      {
+        "token": "content-type",
+        "share_a": 0.083,
+        "share_b": 0.019,
+        "ratio": 4.39
+      },
+      {
+        "token": "official",
+        "share_a": 0.053,
+        "share_b": 0.012,
+        "ratio": 4.38
+      },
+      {
+        "token": "message-id",
+        "share_a": 0.08,
+        "share_b": 0.019,
+        "ratio": 4.28
+      },
+      {
+        "token": "service",
+        "share_a": 0.165,
+        "share_b": 0.041,
+        "ratio": 4.0
+      },
+      {
+        "token": "hosts",
+        "share_a": 0.052,
+        "share_b": 0.013,
+        "ratio": 3.98
+      },
+      {
+        "token": "reply-to",
+        "share_a": 0.05,
+        "share_b": 0.013,
+        "ratio": 3.97
+      },
+      {
+        "token": "sender",
+        "share_a": 0.075,
+        "share_b": 0.019,
+        "ratio": 3.96
+      },
+      {
+        "token": "forum",
+        "share_a": 0.064,
+        "share_b": 0.017,
+        "ratio": 3.79
+      },
+      {
+        "token": "decide",
+        "share_a": 0.049,
+        "share_b": 0.013,
+        "ratio": 3.75
+      },
+      {
+        "token": "us-ascii",
+        "share_a": 0.057,
+        "share_b": 0.015,
+        "ratio": 3.73
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:052869",
+      "primary:080991",
+      "primary:020947",
+      "primary:024445",
+      "primary:017336",
+      "primary:086987",
+      "primary:076023",
+      "primary:199972",
+      "primary:143065",
+      "primary:045125",
+      "primary:145723",
+      "primary:147193",
+      "primary:085220",
+      "primary:186806",
+      "primary:171201",
+      "primary:167537",
+      "primary:119133",
+      "primary:166654",
+      "primary:024565",
+      "primary:177199",
+      "primary:071266",
+      "primary:036450",
+      "primary:121268",
+      "primary:160104",
+      "primary:081278"
+    ],
+    "top_fp_doc_ids": [
+      "primary:143547",
+      "primary:118028",
+      "primary:182736",
+      "primary:118388",
+      "primary:126834",
+      "primary:052736",
+      "primary:065595",
+      "primary:022188",
+      "primary:194751",
+      "primary:068707",
+      "primary:142763",
+      "primary:094404",
+      "primary:041370",
+      "primary:187756",
+      "primary:085043",
+      "primary:049322",
+      "primary:174983",
+      "primary:046160",
+      "primary:158244",
+      "primary:107132",
+      "primary:070866",
+      "primary:075107",
+      "primary:042367",
+      "primary:202546",
+      "primary:010440"
+    ],
+    "threeway": null
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-12L/confusion_matrix.png",
+    "roc_curve": "reports/eval_ft-12L/roc_curve.png",
+    "pr_curve": "reports/eval_ft-12L/pr_curve.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-12L/metrics_test3.json b/presentation/reports/eval_ft-12L/metrics_test3.json
new file mode 100644
index 0000000000000000000000000000000000000000..b0b06fe833f8f6d821d79136fcbee6b353ed5df6
--- /dev/null
+++ b/presentation/reports/eval_ft-12L/metrics_test3.json
@@ -0,0 +1,1680 @@
+{
+  "timestamp": "2026-10-09 08:54",
+  "run_name": "ft-12L",
+  "split": "test3",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout3.parquet",
+  "split_title": "three-way holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-12L/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 12,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 186290179,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-12L",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 12,
+      "freeze_layers": 0,
+      "loss": "cross_entropy",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 1,
+    "best_step": 6250,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9925057415689592,
+    "val_metrics": {
+      "accuracy": 0.9918869405914682,
+      "precision": 0.9935866408518877,
+      "recall": 0.9914271914996378,
+      "f1": 0.9925057415689592,
+      "roc_auc": 0.9995007118744352,
+      "pr_auc": 0.9995737781814353,
+      "tn": 6949,
+      "fp": 53,
+      "fn": 71,
+      "tp": 8211,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.17105858027935028,
+      "f1_at_optimal_threshold": 0.9928829915560917,
+      "f1_unwanted": 0.9925057415689592,
+      "n_3way": 8105,
+      "accuracy_3way": 0.9916101172115978,
+      "macro_f1_3way": 0.9774079655816474,
+      "precision_legitimate": 0.9984195402298851,
+      "recall_legitimate": 0.9924307340759783,
+      "f1_legitimate": 0.9954161294943418,
+      "precision_phishing": 0.9570200573065902,
+      "recall_phishing": 0.9823529411764705,
+      "f1_phishing": 0.969521044992743,
+      "precision_spam": 0.9472361809045227,
+      "recall_spam": 0.9882044560943644,
+      "f1_spam": 0.9672867222578576,
+      "cm3_ll": 6949,
+      "cm3_lp": 13,
+      "cm3_ls": 40,
+      "cm3_pl": 4,
+      "cm3_pp": 334,
+      "cm3_ps": 2,
+      "cm3_sl": 7,
+      "cm3_sp": 2,
+      "cm3_ss": 754,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 338,
+      "pvs_phish_as_spam": 2,
+      "pvs_spam_as_phish": 2,
+      "pvs_spam_as_spam": 761,
+      "val_loss": 0.035248381216093624,
+      "n": 15284,
+      "eval_seconds": 22.8
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9918869405914682,
+        "precision": 0.9935866408518877,
+        "recall": 0.9914271914996378,
+        "f1": 0.9925057415689592,
+        "roc_auc": 0.9995007118744352,
+        "pr_auc": 0.9995737781814353,
+        "tn": 6949,
+        "fp": 53,
+        "fn": 71,
+        "tp": 8211,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.17105858027935028,
+        "f1_at_optimal_threshold": 0.9928829915560917,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.9916101172115978,
+        "macro_f1": 0.9774079655816474,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9984195402298851,
+            "recall": 0.9924307340759783,
+            "f1": 0.9954161294943418
+          },
+          "Phishing": {
+            "precision": 0.9570200573065902,
+            "recall": 0.9823529411764705,
+            "f1": 0.969521044992743
+          },
+          "Spam": {
+            "precision": 0.9472361809045227,
+            "recall": 0.9882044560943644,
+            "f1": 0.9672867222578576
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6949,
+              13,
+              40
+            ],
+            [
+              4,
+              334,
+              2
+            ],
+            [
+              7,
+              2,
+              754
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              338,
+              2
+            ],
+            [
+              2,
+              761
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.17105858027935028,
+      "val_f1_at_optimal": 0.9928829915560917
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 186290179,
+      "trainable": 186290179,
+      "embeddings": 34080768,
+      "encoder_layers": 151154688,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 3220,
+    "positive": 1221,
+    "negative": 1999,
+    "positive_rate": 0.3792,
+    "label3": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "by_source": {
+      "Assassin": {
+        "rows": 236,
+        "positive": 72,
+        "label3": {
+          "legit": 164,
+          "phish": 0,
+          "spam": 72,
+          "unwanted": 0
+        }
+      },
+      "CEAS-08": {
+        "rows": 736,
+        "positive": 0,
+        "label3": {
+          "legit": 736,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Enron": {
+        "rows": 628,
+        "positive": 628,
+        "label3": {
+          "legit": 0,
+          "phish": 0,
+          "spam": 628,
+          "unwanted": 0
+        }
+      },
+      "Nazario": {
+        "rows": 222,
+        "positive": 222,
+        "label3": {
+          "legit": 0,
+          "phish": 222,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Nigerian_Fraud": {
+        "rows": 299,
+        "positive": 299,
+        "label3": {
+          "legit": 0,
+          "phish": 299,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "TREC-07": {
+        "rows": 1099,
+        "positive": 0,
+        "label3": {
+          "legit": 1099,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      }
+    },
+    "mean_tokens_full": 835.2,
+    "pct_truncated": 34.01
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9866459627329193,
+    "precision": 0.985973597359736,
+    "recall": 0.9787059787059788,
+    "f1": 0.9823263460748048,
+    "specificity": 0.991495747873937,
+    "false_positive_rate": 0.008504252126063032,
+    "false_negative_rate": 0.021294021294021293,
+    "confusion_matrix": {
+      "tn": 1982,
+      "fp": 17,
+      "fn": 26,
+      "tp": 1195
+    },
+    "n": 3220,
+    "n_positive": 1221,
+    "n_negative": 1999
+  },
+  "threshold_free": {
+    "roc_auc": 0.9990709933181169,
+    "pr_auc": 0.9983252258043704,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.9795249795249795,
+      "fpr": 0.009504752376188095,
+      "threshold": 0.4533155560493469,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.7747747747747747,
+      "fpr": 0.0005002501250625312,
+      "threshold": 0.9997867345809937,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.16916866600513458,
+      "f1": 0.9848670756646216,
+      "precision": 0.9836601307189542,
+      "recall": 0.9860769860769861
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 1195,
+        "predicted_phishing": 508,
+        "predicted_spam": 687
+      },
+      "false_positives_flagged": {
+        "n": 17,
+        "predicted_phishing": 5,
+        "predicted_spam": 12
+      },
+      "missed_positives": {
+        "n": 26,
+        "predicted_phishing": 4,
+        "predicted_spam": 22
+      },
+      "all_flagged": {
+        "n": 1212,
+        "predicted_phishing": 513,
+        "predicted_spam": 699
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 0,
+        "predicted_phishing": 0,
+        "predicted_spam": 0
+      }
+    },
+    "rows_with_full_label": 3220,
+    "rows_unwanted_only": 0,
+    "evaluated": true,
+    "reason": null,
+    "confusion_matrix": {
+      "labels": [
+        "Legitimate",
+        "Phishing",
+        "Spam"
+      ],
+      "matrix": [
+        [
+          1982,
+          5,
+          12
+        ],
+        [
+          7,
+          507,
+          7
+        ],
+        [
+          19,
+          1,
+          680
+        ]
+      ],
+      "cells": {
+        "true_legitimate": {
+          "pred_legitimate": 1982,
+          "pred_phishing": 5,
+          "pred_spam": 12
+        },
+        "true_phishing": {
+          "pred_legitimate": 7,
+          "pred_phishing": 507,
+          "pred_spam": 7
+        },
+        "true_spam": {
+          "pred_legitimate": 19,
+          "pred_phishing": 1,
+          "pred_spam": 680
+        }
+      }
+    },
+    "accuracy": 0.9841614906832298,
+    "macro_f1": 0.9806831216094775,
+    "macro_f1_present_classes": 0.9806831216094775,
+    "classes_present": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "per_class": {
+      "Legitimate": {
+        "precision": 0.9870517928286853,
+        "recall": 0.991495747873937,
+        "f1": 0.9892687796356376,
+        "support": 1999
+      },
+      "Phishing": {
+        "precision": 0.9883040935672515,
+        "recall": 0.9731285988483686,
+        "f1": 0.9806576402321083,
+        "support": 521
+      },
+      "Spam": {
+        "precision": 0.9728183118741058,
+        "recall": 0.9714285714285714,
+        "f1": 0.9721229449606862,
+        "support": 700
+      }
+    },
+    "argmax_rule": {
+      "doc": "plain argmax over the three probabilities (secondary; can disagree with the binary verdict near the threshold)",
+      "accuracy": 0.9841614906832298,
+      "macro_f1": 0.9806831216094775,
+      "per_class": {
+        "Legitimate": {
+          "precision": 0.9870517928286853,
+          "recall": 0.991495747873937,
+          "f1": 0.9892687796356376,
+          "support": 1999
+        },
+        "Phishing": {
+          "precision": 0.9883040935672515,
+          "recall": 0.9731285988483686,
+          "f1": 0.9806576402321083,
+          "support": 521
+        },
+        "Spam": {
+          "precision": 0.9728183118741058,
+          "recall": 0.9714285714285714,
+          "f1": 0.9721229449606862,
+          "support": 700
+        }
+      },
+      "rows_where_rules_differ": 0
+    },
+    "phish_vs_spam_on_true_unwanted": {
+      "n_true_unwanted": 1221,
+      "kind_only": {
+        "doc": "given the row is unwanted, is argmax(P(phishing), P(spam)) right? (independent of the roll-up threshold)",
+        "labels": [
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            511,
+            10
+          ],
+          [
+            1,
+            699
+          ]
+        ],
+        "cells": {
+          "true_phishing": {
+            "pred_phishing": 511,
+            "pred_spam": 10
+          },
+          "true_spam": {
+            "pred_phishing": 1,
+            "pred_spam": 699
+          }
+        },
+        "accuracy": 0.990990990990991,
+        "phishing_as_spam": 10,
+        "spam_as_phishing": 1
+      },
+      "rollup_rule": {
+        "doc": "rows = true Phishing / Spam; columns = predicted Legitimate (missed) / Phishing / Spam under the roll-up rule",
+        "rows": [
+          "Phishing",
+          "Spam"
+        ],
+        "columns": [
+          "Legitimate",
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            7,
+            507,
+            7
+          ],
+          [
+            19,
+            1,
+            680
+          ]
+        ],
+        "missed_as_legitimate": {
+          "phishing": 7,
+          "spam": 19
+        }
+      }
+    },
+    "per_source": {
+      "Assassin": {
+        "n": 236,
+        "counts": {
+          "Legitimate": 164,
+          "Phishing": 0,
+          "Spam": 72
+        },
+        "accuracy": 0.961864406779661,
+        "macro_f1": 0.6370518043396608,
+        "macro_f1_present_classes": 0.9555777065094913,
+        "per_class_recall": {
+          "Legitimate": 0.9878048780487805,
+          "Spam": 0.9027777777777778
+        }
+      },
+      "CEAS-08": {
+        "n": 736,
+        "counts": {
+          "Legitimate": 736,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9864130434782609,
+        "macro_f1": 0.33105335157318744,
+        "macro_f1_present_classes": 0.9931600547195623,
+        "per_class_recall": {
+          "Legitimate": 0.9864130434782609
+        }
+      },
+      "Enron": {
+        "n": 628,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 0,
+          "Spam": 628
+        },
+        "accuracy": 0.9792993630573248,
+        "macro_f1": 0.32984714400643605,
+        "macro_f1_present_classes": 0.9895414320193081,
+        "per_class_recall": {
+          "Spam": 0.9792993630573248
+        }
+      },
+      "Nazario": {
+        "n": 222,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 222,
+          "Spam": 0
+        },
+        "accuracy": 0.9684684684684685,
+        "macro_f1": 0.32799389778794813,
+        "macro_f1_present_classes": 0.9839816933638444,
+        "per_class_recall": {
+          "Phishing": 0.9684684684684685
+        }
+      },
+      "Nigerian_Fraud": {
+        "n": 299,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 299,
+          "Spam": 0
+        },
+        "accuracy": 0.9765886287625418,
+        "macro_f1": 0.329385222786238,
+        "macro_f1_present_classes": 0.988155668358714,
+        "per_class_recall": {
+          "Phishing": 0.9765886287625418
+        }
+      },
+      "TREC-07": {
+        "n": 1099,
+        "counts": {
+          "Legitimate": 1099,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9954504094631483,
+        "macro_f1": 0.3325733394132847,
+        "macro_f1_present_classes": 0.9977200182398541,
+        "per_class_recall": {
+          "Legitimate": 0.9954504094631483
+        }
+      }
+    }
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 139,
+      "bank account": 91,
+      "urgently": 25,
+      "verify your account": 15,
+      "confirm your account": 9,
+      "update your account": 9,
+      "within 24 hours": 8,
+      "security alert": 8,
+      "paypal account": 7,
+      "password will expire": 7,
+      "click here to verify": 6,
+      "unusual activity": 4,
+      "wire transfer": 3,
+      "update your information": 3,
+      "gift card": 3,
+      "verify your identity": 2,
+      "unusual sign-in": 2,
+      "account will be suspended": 2,
+      "update your payment": 2,
+      "failure to verify": 2,
+      "within 48 hours": 2,
+      "confirm your identity": 2,
+      "enter your password": 1,
+      "update your billing": 1,
+      "reset your password": 1,
+      "suspicious activity": 1,
+      "security notice": 1,
+      "log in to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 15,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "update your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 2,
+        "negative": 1
+      },
+      "update your billing": {
+        "positive": 1,
+        "negative": 0
+      },
+      "update your payment": {
+        "positive": 2,
+        "negative": 0
+      },
+      "click here to verify": {
+        "positive": 6,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual sign-in": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "suspicious activity": {
+        "positive": 1,
+        "negative": 0
+      },
+      "paypal account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "password will expire": {
+        "positive": 7,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 1,
+        "negative": 0
+      },
+      "wire transfer": {
+        "positive": 2,
+        "negative": 1
+      },
+      "gift card": {
+        "positive": 3,
+        "negative": 0
+      },
+      "urgent": {
+        "positive": 133,
+        "negative": 6
+      },
+      "urgently": {
+        "positive": 25,
+        "negative": 0
+      },
+      "security alert": {
+        "positive": 8,
+        "negative": 0
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 91,
+        "negative": 0
+      },
+      "failure to verify": {
+        "positive": 2,
+        "negative": 0
+      },
+      "within 24 hours": {
+        "positive": 8,
+        "negative": 0
+      },
+      "within 48 hours": {
+        "positive": 2,
+        "negative": 0
+      }
+    },
+    "label_purity": 0.9745,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 7,
+      "phish": 236,
+      "spam": 32,
+      "unwanted": 0
+    },
+    "threeway_accuracy": 0.9781818181818182,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 0.9890909090909091,
+      "precision": 1.0,
+      "recall": 0.9888059701492538,
+      "f1": 0.9943714821763603,
+      "specificity": 1.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.011194029850746268,
+      "confusion_matrix": {
+        "tn": 7,
+        "fp": 0,
+        "fn": 3,
+        "tp": 265
+      },
+      "n": 275,
+      "n_positive": 268,
+      "n_negative": 7,
+      "roc_auc": 1.0
+    }
+  },
+  "per_source": {
+    "Assassin": {
+      "threshold": 0.5,
+      "accuracy": 0.9661016949152542,
+      "precision": 0.9705882352941176,
+      "recall": 0.9166666666666666,
+      "f1": 0.9428571428571428,
+      "specificity": 0.9878048780487805,
+      "false_positive_rate": 0.012195121951219513,
+      "false_negative_rate": 0.08333333333333333,
+      "confusion_matrix": {
+        "tn": 162,
+        "fp": 2,
+        "fn": 6,
+        "tp": 66
+      },
+      "n": 236,
+      "n_positive": 72,
+      "n_negative": 164,
+      "roc_auc": 0.9961890243902439,
+      "label3_counts": {
+        "legit": 164,
+        "phish": 0,
+        "spam": 72,
+        "unwanted": 0
+      }
+    },
+    "CEAS-08": {
+      "threshold": 0.5,
+      "accuracy": 0.9864130434782609,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9864130434782609,
+      "false_positive_rate": 0.01358695652173913,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 726,
+        "fp": 10,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 736,
+      "n_positive": 0,
+      "n_negative": 736,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 736,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Enron": {
+      "threshold": 0.5,
+      "accuracy": 0.9792993630573248,
+      "precision": 1.0,
+      "recall": 0.9792993630573248,
+      "f1": 0.9895414320193081,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.020700636942675158,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 13,
+        "tp": 615
+      },
+      "n": 628,
+      "n_positive": 628,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 0,
+        "spam": 628,
+        "unwanted": 0
+      }
+    },
+    "Nazario": {
+      "threshold": 0.5,
+      "accuracy": 0.9819819819819819,
+      "precision": 1.0,
+      "recall": 0.9819819819819819,
+      "f1": 0.9909090909090909,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.018018018018018018,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 4,
+        "tp": 218
+      },
+      "n": 222,
+      "n_positive": 222,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 222,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Nigerian_Fraud": {
+      "threshold": 0.5,
+      "accuracy": 0.9899665551839465,
+      "precision": 1.0,
+      "recall": 0.9899665551839465,
+      "f1": 0.9949579831932773,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.010033444816053512,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 3,
+        "tp": 296
+      },
+      "n": 299,
+      "n_positive": 299,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 299,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "TREC-07": {
+      "threshold": 0.5,
+      "accuracy": 0.9954504094631483,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9954504094631483,
+      "false_positive_rate": 0.004549590536851683,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 1094,
+        "fp": 5,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 1099,
+      "n_positive": 0,
+      "n_negative": 1099,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 1099,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 1195,
+        "mean_tokens_full": 1003.6,
+        "mean_tokens_fed": 280.3,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.5
+      },
+      "tn": {
+        "n": 1982,
+        "mean_tokens_full": 733.4,
+        "mean_tokens_fed": 358.3,
+        "median_tokens_full": 392.5,
+        "pct_truncated": 38.3
+      },
+      "fp": {
+        "n": 17,
+        "mean_tokens_full": 1469.5,
+        "mean_tokens_fed": 384.9,
+        "median_tokens_full": 875.0,
+        "pct_truncated": 58.8
+      },
+      "fn": {
+        "n": 26,
+        "mean_tokens_full": 443.6,
+        "mean_tokens_fed": 260.9,
+        "median_tokens_full": 177.0,
+        "pct_truncated": 34.6
+      },
+      "correct": {
+        "n": 3177,
+        "mean_tokens_full": 835.0,
+        "mean_tokens_fed": 329.0,
+        "median_tokens_full": 339.0,
+        "pct_truncated": 33.9
+      },
+      "wrong": {
+        "n": 43,
+        "mean_tokens_full": 849.2,
+        "mean_tokens_fed": 309.9,
+        "median_tokens_full": 340.0,
+        "pct_truncated": 44.2
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "unwanted": {
+        "n": 1221,
+        "mean_tokens_full": 991.6,
+        "mean_tokens_fed": 279.9,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.7
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "phish": {
+        "n": 521,
+        "mean_tokens_full": 1349.4,
+        "mean_tokens_fed": 355.1,
+        "median_tokens_full": 412.0,
+        "pct_truncated": 39.9
+      },
+      "spam": {
+        "n": 700,
+        "mean_tokens_full": 725.3,
+        "mean_tokens_fed": 223.9,
+        "median_tokens_full": 159.0,
+        "pct_truncated": 16.9
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 129,
+        "n_positive": 99,
+        "positive_rate": 0.7674418604651163,
+        "recall": 0.9393939393939394,
+        "fpr": 0.03333333333333333,
+        "accuracy": 0.9457364341085271
+      },
+      {
+        "bucket": "65-128",
+        "n": 427,
+        "n_positive": 252,
+        "positive_rate": 0.5901639344262295,
+        "recall": 0.9880952380952381,
+        "fpr": 0.005714285714285714,
+        "accuracy": 0.990632318501171
+      },
+      {
+        "bucket": "129-256",
+        "n": 699,
+        "n_positive": 276,
+        "positive_rate": 0.3948497854077253,
+        "recall": 0.9818840579710145,
+        "fpr": 0.004728132387706856,
+        "accuracy": 0.9899856938483548
+      },
+      {
+        "bucket": "257-512",
+        "n": 870,
+        "n_positive": 268,
+        "positive_rate": 0.3080459770114943,
+        "recall": 0.9888059701492538,
+        "fpr": 0.0049833887043189366,
+        "accuracy": 0.993103448275862
+      },
+      {
+        "bucket": ">512",
+        "n": 1095,
+        "n_positive": 326,
+        "positive_rate": 0.29771689497716897,
+        "recall": 0.9723926380368099,
+        "fpr": 0.013003901170351105,
+        "accuracy": 0.982648401826484
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 3220,
+    "seconds": 6.42,
+    "rows_per_s": 501.6,
+    "padded_tokens": 1079936,
+    "real_tokens": 1058490,
+    "padded_tokens_per_s": 168244,
+    "peak_vram_gb": 1.7
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 5.05,
+      "p50_ms": 4.42,
+      "p95_ms": 6.98,
+      "max_ms": 12.33,
+      "avg_s": 0.0051,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.068,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 246.62,
+      "p50_ms": 242.67,
+      "p95_ms": 356.11,
+      "max_ms": 389.87,
+      "avg_s": 0.2466,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 3.333,
+      "threads": 16,
+      "estimated_full_run_s": 27.1
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-12L/best",
+    "run_name": "ft-12L",
+    "params": 186290179,
+    "timestamp": "2026-10-09 08:54",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-12L/error_analysis_test3.md",
+    "n_fn": 26,
+    "n_fp": 17,
+    "composition": {
+      "fp": {
+        "n": 17,
+        "mailing_list_or_digest": 0.588,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.588,
+        "lure_phrase": 0.0
+      },
+      "tn": {
+        "n": 1982,
+        "mailing_list_or_digest": 0.563,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.383,
+        "lure_phrase": 0.004
+      },
+      "fn": {
+        "n": 26,
+        "mailing_list_or_digest": 0.269,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.115,
+        "truncated_gt512": 0.346,
+        "lure_phrase": 0.115
+      },
+      "tp": {
+        "n": 1195,
+        "mailing_list_or_digest": 0.083,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.005,
+        "truncated_gt512": 0.265,
+        "lure_phrase": 0.222
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "invite",
+        "share_a": 0.115,
+        "share_b": 0.003,
+        "ratio": 35.09
+      },
+      {
+        "token": "oct",
+        "share_a": 0.115,
+        "share_b": 0.003,
+        "ratio": 35.09
+      },
+      {
+        "token": "association",
+        "share_a": 0.192,
+        "share_b": 0.007,
+        "ratio": 29.19
+      },
+      {
+        "token": "angeles",
+        "share_a": 0.115,
+        "share_b": 0.004,
+        "ratio": 28.71
+      },
+      {
+        "token": "xent",
+        "share_a": 0.115,
+        "share_b": 0.004,
+        "ratio": 28.71
+      },
+      {
+        "token": "fork",
+        "share_a": 0.115,
+        "share_b": 0.004,
+        "ratio": 28.71
+      },
+      {
+        "token": "mailman",
+        "share_a": 0.192,
+        "share_b": 0.008,
+        "ratio": 26.12
+      },
+      {
+        "token": "environment",
+        "share_a": 0.115,
+        "share_b": 0.005,
+        "ratio": 24.29
+      },
+      {
+        "token": "york",
+        "share_a": 0.115,
+        "share_b": 0.005,
+        "ratio": 24.29
+      },
+      {
+        "token": "community",
+        "share_a": 0.115,
+        "share_b": 0.005,
+        "ratio": 24.29
+      },
+      {
+        "token": "listinfo",
+        "share_a": 0.231,
+        "share_b": 0.01,
+        "ratio": 23.46
+      },
+      {
+        "token": "subscription",
+        "share_a": 0.154,
+        "share_b": 0.008,
+        "ratio": 21.37
+      },
+      {
+        "token": "event",
+        "share_a": 0.115,
+        "share_b": 0.006,
+        "ratio": 21.05
+      },
+      {
+        "token": "technologies",
+        "share_a": 0.115,
+        "share_b": 0.006,
+        "ratio": 21.05
+      },
+      {
+        "token": "eleven",
+        "share_a": 0.115,
+        "share_b": 0.006,
+        "ratio": 21.05
+      },
+      {
+        "token": "entertainment",
+        "share_a": 0.115,
+        "share_b": 0.007,
+        "ratio": 18.58
+      },
+      {
+        "token": "wonderful",
+        "share_a": 0.115,
+        "share_b": 0.007,
+        "ratio": 18.58
+      },
+      {
+        "token": "wants",
+        "share_a": 0.115,
+        "share_b": 0.008,
+        "ratio": 16.62
+      },
+      {
+        "token": "wanted",
+        "share_a": 0.115,
+        "share_b": 0.009,
+        "ratio": 13.73
+      },
+      {
+        "token": "sound",
+        "share_a": 0.115,
+        "share_b": 0.011,
+        "ratio": 11.7
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "heid",
+        "share_a": 0.176,
+        "share_b": 0.0,
+        "ratio": 793.0
+      },
+      {
+        "token": "dreamweaver",
+        "share_a": 0.176,
+        "share_b": 0.0,
+        "ratio": 793.0
+      },
+      {
+        "token": "stylesheets",
+        "share_a": 0.176,
+        "share_b": 0.0,
+        "ratio": 793.0
+      },
+      {
+        "token": "rias",
+        "share_a": 0.176,
+        "share_b": 0.001,
+        "ratio": 264.33
+      },
+      {
+        "token": "ears",
+        "share_a": 0.176,
+        "share_b": 0.001,
+        "ratio": 158.6
+      },
+      {
+        "token": "stewart",
+        "share_a": 0.176,
+        "share_b": 0.001,
+        "ratio": 158.6
+      },
+      {
+        "token": "untitled",
+        "share_a": 0.176,
+        "share_b": 0.001,
+        "ratio": 158.6
+      },
+      {
+        "token": "handheld",
+        "share_a": 0.176,
+        "share_b": 0.001,
+        "ratio": 158.6
+      },
+      {
+        "token": "css",
+        "share_a": 0.176,
+        "share_b": 0.002,
+        "ratio": 113.29
+      },
+      {
+        "token": "keynote",
+        "share_a": 0.176,
+        "share_b": 0.002,
+        "ratio": 113.29
+      },
+      {
+        "token": "inspire",
+        "share_a": 0.176,
+        "share_b": 0.002,
+        "ratio": 113.29
+      },
+      {
+        "token": "dollars",
+        "share_a": 0.235,
+        "share_b": 0.003,
+        "ratio": 92.69
+      },
+      {
+        "token": "rock",
+        "share_a": 0.235,
+        "share_b": 0.003,
+        "ratio": 92.69
+      },
+      {
+        "token": "ajax",
+        "share_a": 0.176,
+        "share_b": 0.002,
+        "ratio": 88.11
+      },
+      {
+        "token": "ridge",
+        "share_a": 0.176,
+        "share_b": 0.002,
+        "ratio": 88.11
+      },
+      {
+        "token": "commitment",
+        "share_a": 0.176,
+        "share_b": 0.003,
+        "ratio": 72.09
+      },
+      {
+        "token": "ted",
+        "share_a": 0.176,
+        "share_b": 0.003,
+        "ratio": 72.09
+      },
+      {
+        "token": "front-end",
+        "share_a": 0.176,
+        "share_b": 0.003,
+        "ratio": 72.09
+      },
+      {
+        "token": "historic",
+        "share_a": 0.176,
+        "share_b": 0.003,
+        "ratio": 61.0
+      },
+      {
+        "token": "adobe",
+        "share_a": 0.176,
+        "share_b": 0.003,
+        "ratio": 61.0
+      }
+    ],
+    "top_fn_doc_ids": [
+      "aura:Nigerian_Fraud:001032",
+      "primary:007176",
+      "aura:Nazario:000097",
+      "primary:112100",
+      "primary:001939",
+      "primary:148887",
+      "primary:135631",
+      "aura:Nazario:000643",
+      "primary:010013",
+      "primary:102650",
+      "aura:Nazario:001540",
+      "primary:025244",
+      "primary:148857",
+      "aura:Nazario:000551",
+      "primary:061572",
+      "primary:107809",
+      "primary:046569",
+      "primary:181246",
+      "aura:Nigerian_Fraud:000534",
+      "aura:Nigerian_Fraud:000495",
+      "primary:025024",
+      "primary:150100",
+      "primary:191966",
+      "primary:157116",
+      "primary:197304"
+    ],
+    "top_fp_doc_ids": [
+      "primary:144048",
+      "primary:164457",
+      "primary:153841",
+      "primary:091801",
+      "primary:156985",
+      "primary:003448",
+      "primary:041713",
+      "primary:152763",
+      "primary:156887",
+      "primary:057514",
+      "primary:199386",
+      "primary:055965",
+      "primary:168120",
+      "primary:024251",
+      "primary:089923",
+      "primary:016587",
+      "primary:021911"
+    ],
+    "threeway": {
+      "n_phish_as_spam": 7,
+      "n_spam_as_phish": 1,
+      "top_phish_as_spam_doc_ids": [
+        "aura:Nazario:000532",
+        "aura:Nigerian_Fraud:000824",
+        "aura:Nazario:000328",
+        "aura:Nigerian_Fraud:001698",
+        "aura:Nigerian_Fraud:003049",
+        "aura:Nigerian_Fraud:000337",
+        "aura:Nazario:000401"
+      ],
+      "top_spam_as_phish_doc_ids": [
+        "primary:077835"
+      ],
+      "phish_as_spam_tokens": [
+        {
+          "token": "join",
+          "share_a": 0.571,
+          "share_b": 0.02,
+          "ratio": 29.0
+        },
+        {
+          "token": "friends",
+          "share_a": 0.429,
+          "share_b": 0.016,
+          "ratio": 27.86
+        },
+        {
+          "token": "complete",
+          "share_a": 0.429,
+          "share_b": 0.049,
+          "ratio": 9.29
+        },
+        {
+          "token": "link",
+          "share_a": 0.571,
+          "share_b": 0.073,
+          "ratio": 8.12
+        },
+        {
+          "token": "don't",
+          "share_a": 0.429,
+          "share_b": 0.067,
+          "ratio": 6.86
+        },
+        {
+          "token": "could",
+          "share_a": 0.429,
+          "share_b": 0.077,
+          "ratio": 6.0
+        },
+        {
+          "token": "http",
+          "share_a": 0.857,
+          "share_b": 0.189,
+          "ratio": 4.56
+        },
+        {
+          "token": "www",
+          "share_a": 0.571,
+          "share_b": 0.15,
+          "ratio": 3.98
+        },
+        {
+          "token": "new",
+          "share_a": 0.429,
+          "share_b": 0.164,
+          "ratio": 2.84
+        },
+        {
+          "token": "click",
+          "share_a": 0.429,
+          "share_b": 0.176,
+          "ratio": 2.65
+        },
+        {
+          "token": "years",
+          "share_a": 0.429,
+          "share_b": 0.195,
+          "ratio": 2.38
+        },
+        {
+          "token": "get",
+          "share_a": 0.429,
+          "share_b": 0.203,
+          "ratio": 2.29
+        },
+        {
+          "token": "com",
+          "share_a": 0.714,
+          "share_b": 0.335,
+          "ratio": 2.18
+        },
+        {
+          "token": "message",
+          "share_a": 0.429,
+          "share_b": 0.213,
+          "ratio": 2.18
+        },
+        {
+          "token": "about",
+          "share_a": 0.429,
+          "share_b": 0.241,
+          "ratio": 1.93
+        },
+        {
+          "token": "who",
+          "share_a": 0.571,
+          "share_b": 0.316,
+          "ratio": 1.9
+        },
+        {
+          "token": "after",
+          "share_a": 0.429,
+          "share_b": 0.278,
+          "ratio": 1.67
+        },
+        {
+          "token": "are",
+          "share_a": 0.857,
+          "share_b": 0.562,
+          "ratio": 1.54
+        },
+        {
+          "token": "into",
+          "share_a": 0.429,
+          "share_b": 0.325,
+          "ratio": 1.43
+        },
+        {
+          "token": "which",
+          "share_a": 0.429,
+          "share_b": 0.361,
+          "ratio": 1.29
+        }
+      ],
+      "spam_as_phish_tokens": []
+    }
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-12L/confusion_matrix_test3.png",
+    "confusion_matrix3": "reports/eval_ft-12L/confusion_matrix3_test3.png",
+    "roc_curve": "reports/eval_ft-12L/roc_curve_test3.png",
+    "pr_curve": "reports/eval_ft-12L/pr_curve_test3.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-ce/metrics.json b/presentation/reports/eval_ft-ce/metrics.json
new file mode 100644
index 0000000000000000000000000000000000000000..de12d5e04a5329b3329612736857c5f9dbe73d69
--- /dev/null
+++ b/presentation/reports/eval_ft-ce/metrics.json
@@ -0,0 +1,1109 @@
+{
+  "timestamp": "2026-10-09 04:28",
+  "run_name": "ft-ce",
+  "split": "test",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet",
+  "split_title": "TREC-06 binary holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-ce/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-ce",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "cross_entropy",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 3,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 0,
+    "best_step": 5000,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.992930086410055,
+    "val_metrics": {
+      "accuracy": 0.9923449358806595,
+      "precision": 0.9938308939155679,
+      "recall": 0.992030910408114,
+      "f1": 0.992930086410055,
+      "roc_auc": 0.9995503578823617,
+      "pr_auc": 0.9996107014519675,
+      "tn": 6951,
+      "fp": 51,
+      "fn": 66,
+      "tp": 8216,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.31750011444091797,
+      "f1_at_optimal_threshold": 0.9931808581256412,
+      "f1_unwanted": 0.992930086410055,
+      "n_3way": 8105,
+      "accuracy_3way": 0.9922270203578039,
+      "macro_f1_3way": 0.9781799660145108,
+      "precision_legitimate": 0.9988504095416008,
+      "recall_legitimate": 0.9927163667523564,
+      "f1_legitimate": 0.995773941694721,
+      "precision_phishing": 0.9390581717451524,
+      "recall_phishing": 0.9970588235294118,
+      "f1_phishing": 0.9671897289586305,
+      "precision_spam": 0.9579617834394905,
+      "recall_spam": 0.9855832241153342,
+      "f1_spam": 0.9715762273901809,
+      "cm3_ll": 6951,
+      "cm3_lp": 18,
+      "cm3_ls": 33,
+      "cm3_pl": 1,
+      "cm3_pp": 339,
+      "cm3_ps": 0,
+      "cm3_sl": 7,
+      "cm3_sp": 4,
+      "cm3_ss": 752,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 340,
+      "pvs_phish_as_spam": 0,
+      "pvs_spam_as_phish": 4,
+      "pvs_spam_as_spam": 759,
+      "val_loss": 0.027098098741997853,
+      "n": 15284,
+      "eval_seconds": 47.7
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9923449358806595,
+        "precision": 0.9938308939155679,
+        "recall": 0.992030910408114,
+        "f1": 0.992930086410055,
+        "roc_auc": 0.9995503578823617,
+        "pr_auc": 0.9996107014519675,
+        "tn": 6951,
+        "fp": 51,
+        "fn": 66,
+        "tp": 8216,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.31750011444091797,
+        "f1_at_optimal_threshold": 0.9931808581256412,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.9922270203578039,
+        "macro_f1": 0.9781799660145108,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9988504095416008,
+            "recall": 0.9927163667523564,
+            "f1": 0.995773941694721
+          },
+          "Phishing": {
+            "precision": 0.9390581717451524,
+            "recall": 0.9970588235294118,
+            "f1": 0.9671897289586305
+          },
+          "Spam": {
+            "precision": 0.9579617834394905,
+            "recall": 0.9855832241153342,
+            "f1": 0.9715762273901809
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6951,
+              18,
+              33
+            ],
+            [
+              1,
+              339,
+              0
+            ],
+            [
+              7,
+              4,
+              752
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              340,
+              0
+            ],
+            [
+              4,
+              759
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.31750011444091797,
+      "val_f1_at_optimal": 0.9931808581256412
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 16302,
+    "positive": 3894,
+    "negative": 12408,
+    "positive_rate": 0.2389,
+    "label3": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "by_source": {
+      "TREC-06": {
+        "rows": 16302,
+        "positive": 3894,
+        "label3": {
+          "legit": 12408,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 3894
+        }
+      }
+    },
+    "mean_tokens_full": 710.6,
+    "pct_truncated": 21.93
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9495153968838179,
+    "precision": 0.8610392664001881,
+    "recall": 0.9404211607601438,
+    "f1": 0.8989812200810113,
+    "specificity": 0.9523694390715667,
+    "false_positive_rate": 0.04763056092843327,
+    "false_negative_rate": 0.05957883923985619,
+    "confusion_matrix": {
+      "tn": 11817,
+      "fp": 591,
+      "fn": 232,
+      "tp": 3662
+    },
+    "n": 16302,
+    "n_positive": 3894,
+    "n_negative": 12408
+  },
+  "threshold_free": {
+    "roc_auc": 0.9874951859346837,
+    "pr_auc": 0.9625515458938435,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.7745249101181304,
+      "fpr": 0.009993552546744036,
+      "threshold": 0.9965154528617859,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.35285053929121724,
+      "fpr": 0.0009671179883945841,
+      "threshold": 0.9997650384902954,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.8211185932159424,
+      "f1": 0.9089527668734962,
+      "precision": 0.8965775668248813,
+      "recall": 0.9216743708269132
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 3662,
+        "predicted_phishing": 1294,
+        "predicted_spam": 2368
+      },
+      "false_positives_flagged": {
+        "n": 591,
+        "predicted_phishing": 374,
+        "predicted_spam": 217
+      },
+      "missed_positives": {
+        "n": 232,
+        "predicted_phishing": 64,
+        "predicted_spam": 168
+      },
+      "all_flagged": {
+        "n": 4253,
+        "predicted_phishing": 1668,
+        "predicted_spam": 2585
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 3662,
+        "predicted_phishing": 1294,
+        "predicted_spam": 2368
+      }
+    },
+    "rows_with_full_label": 12408,
+    "rows_unwanted_only": 3894,
+    "evaluated": false,
+    "reason": "no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)"
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 129,
+      "bank account": 78,
+      "security alert": 70,
+      "within 24 hours": 65,
+      "urgently": 51,
+      "paypal account": 20,
+      "within 48 hours": 7,
+      "verify your account": 7,
+      "update your account": 6,
+      "account will be suspended": 5,
+      "ebay account": 5,
+      "enter your password": 5,
+      "wire transfer": 5,
+      "unusual activity": 5,
+      "confirm your identity": 4,
+      "verify your identity": 4,
+      "click here to verify": 3,
+      "log in to your account": 2,
+      "confirm your account": 2,
+      "security notice": 1,
+      "gift card": 1,
+      "reset your password": 1,
+      "account has been locked": 1,
+      "update your information": 1,
+      "account has been limited": 1,
+      "login to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 5,
+        "negative": 0
+      },
+      "update your account": {
+        "positive": 6,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 0,
+        "negative": 1
+      },
+      "click here to verify": {
+        "positive": 3,
+        "negative": 0
+      },
+      "account has been limited": {
+        "positive": 1,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 4,
+        "negative": 1
+      },
+      "account has been locked": {
+        "positive": 1,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 2,
+        "negative": 3
+      },
+      "paypal account": {
+        "positive": 20,
+        "negative": 0
+      },
+      "ebay account": {
+        "positive": 5,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "wire transfer": {
+        "positive": 5,
+        "negative": 0
+      },
+      "gift card": {
+        "positive": 0,
+        "negative": 1
+      },
+      "urgent": {
+        "positive": 81,
+        "negative": 48
+      },
+      "urgently": {
+        "positive": 22,
+        "negative": 29
+      },
+      "security alert": {
+        "positive": 2,
+        "negative": 68
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "login to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 71,
+        "negative": 7
+      },
+      "within 24 hours": {
+        "positive": 63,
+        "negative": 2
+      },
+      "within 48 hours": {
+        "positive": 4,
+        "negative": 3
+      }
+    },
+    "label_purity": 0.6106,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 155,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 243
+    },
+    "threeway_accuracy": null,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 0.9296482412060302,
+      "precision": 0.8996282527881041,
+      "recall": 0.9958847736625515,
+      "f1": 0.9453124999999999,
+      "specificity": 0.8258064516129032,
+      "false_positive_rate": 0.17419354838709677,
+      "false_negative_rate": 0.00411522633744856,
+      "confusion_matrix": {
+        "tn": 128,
+        "fp": 27,
+        "fn": 1,
+        "tp": 242
+      },
+      "n": 398,
+      "n_positive": 243,
+      "n_negative": 155,
+      "roc_auc": 0.9980087614496216
+    }
+  },
+  "per_source": {
+    "TREC-06": {
+      "threshold": 0.5,
+      "accuracy": 0.9495153968838179,
+      "precision": 0.8610392664001881,
+      "recall": 0.9404211607601438,
+      "f1": 0.8989812200810113,
+      "specificity": 0.9523694390715667,
+      "false_positive_rate": 0.04763056092843327,
+      "false_negative_rate": 0.05957883923985619,
+      "confusion_matrix": {
+        "tn": 11817,
+        "fp": 591,
+        "fn": 232,
+        "tp": 3662
+      },
+      "n": 16302,
+      "n_positive": 3894,
+      "n_negative": 12408,
+      "roc_auc": 0.9874951859346837,
+      "label3_counts": {
+        "legit": 12408,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 3894
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 3662,
+        "mean_tokens_full": 286.7,
+        "mean_tokens_fed": 234.3,
+        "median_tokens_full": 211.0,
+        "pct_truncated": 15.2
+      },
+      "tn": {
+        "n": 11817,
+        "mean_tokens_full": 848.2,
+        "mean_tokens_fed": 293.0,
+        "median_tokens_full": 266.0,
+        "pct_truncated": 23.6
+      },
+      "fp": {
+        "n": 591,
+        "mean_tokens_full": 763.4,
+        "mean_tokens_fed": 306.9,
+        "median_tokens_full": 321.0,
+        "pct_truncated": 31.5
+      },
+      "fn": {
+        "n": 232,
+        "mean_tokens_full": 256.2,
+        "mean_tokens_fed": 194.3,
+        "median_tokens_full": 121.5,
+        "pct_truncated": 16.8
+      },
+      "correct": {
+        "n": 15479,
+        "mean_tokens_full": 715.4,
+        "mean_tokens_fed": 279.1,
+        "median_tokens_full": 246.0,
+        "pct_truncated": 21.6
+      },
+      "wrong": {
+        "n": 823,
+        "mean_tokens_full": 620.4,
+        "mean_tokens_fed": 275.1,
+        "median_tokens_full": 237.0,
+        "pct_truncated": 27.3
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 1363,
+        "n_positive": 587,
+        "positive_rate": 0.43066764490095377,
+        "recall": 0.9045996592844975,
+        "fpr": 0.08376288659793814,
+        "accuracy": 0.9112252384446075
+      },
+      {
+        "bucket": "65-128",
+        "n": 2538,
+        "n_positive": 736,
+        "positive_rate": 0.2899921197793538,
+        "recall": 0.90625,
+        "fpr": 0.039400665926748055,
+        "accuracy": 0.9448384554767534
+      },
+      {
+        "bucket": "129-256",
+        "n": 4594,
+        "n_positive": 1196,
+        "positive_rate": 0.260339573356552,
+        "recall": 0.9565217391304348,
+        "fpr": 0.03384343731606827,
+        "accuracy": 0.9636482368306487
+      },
+      {
+        "bucket": "257-512",
+        "n": 4232,
+        "n_positive": 780,
+        "positive_rate": 0.1843100189035917,
+        "recall": 0.9794871794871794,
+        "fpr": 0.044611819235225954,
+        "accuracy": 0.9598298676748582
+      },
+      {
+        "bucket": ">512",
+        "n": 3575,
+        "n_positive": 595,
+        "positive_rate": 0.16643356643356644,
+        "recall": 0.934453781512605,
+        "fpr": 0.06241610738255034,
+        "accuracy": 0.9370629370629371
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 16302,
+    "seconds": 57.19,
+    "rows_per_s": 285.1,
+    "padded_tokens": 4606544,
+    "real_tokens": 4546874,
+    "padded_tokens_per_s": 80548,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 9.95,
+      "p50_ms": 8.72,
+      "p95_ms": 13.58,
+      "max_ms": 19.1,
+      "avg_s": 0.0099,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.134,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 514.07,
+      "p50_ms": 526.19,
+      "p95_ms": 724.38,
+      "max_ms": 783.31,
+      "avg_s": 0.5141,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 6.947,
+      "threads": 16,
+      "estimated_full_run_s": 51.3
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-ce/best",
+    "run_name": "ft-ce",
+    "params": 337444867,
+    "timestamp": "2026-10-09 04:28",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-ce/error_analysis.md",
+    "n_fn": 232,
+    "n_fp": 591,
+    "composition": {
+      "fp": {
+        "n": 591,
+        "mailing_list_or_digest": 0.284,
+        "html_heavy": 0.002,
+        "very_short_lt20_tokens": 0.024,
+        "truncated_gt512": 0.315,
+        "lure_phrase": 0.046
+      },
+      "tn": {
+        "n": 11817,
+        "mailing_list_or_digest": 0.293,
+        "html_heavy": 0.002,
+        "very_short_lt20_tokens": 0.004,
+        "truncated_gt512": 0.236,
+        "lure_phrase": 0.011
+      },
+      "fn": {
+        "n": 232,
+        "mailing_list_or_digest": 0.043,
+        "html_heavy": 0.004,
+        "very_short_lt20_tokens": 0.004,
+        "truncated_gt512": 0.168,
+        "lure_phrase": 0.004
+      },
+      "tp": {
+        "n": 3662,
+        "mailing_list_or_digest": 0.029,
+        "html_heavy": 0.001,
+        "very_short_lt20_tokens": 0.002,
+        "truncated_gt512": 0.152,
+        "lure_phrase": 0.066
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "blocki",
+        "share_a": 0.052,
+        "share_b": 0.0,
+        "ratio": 393.82
+      },
+      {
+        "token": "scanmail",
+        "share_a": 0.052,
+        "share_b": 0.0,
+        "ratio": 393.82
+      },
+      {
+        "token": "smailex",
+        "share_a": 0.052,
+        "share_b": 0.0,
+        "ratio": 393.82
+      },
+      {
+        "token": "progra",
+        "share_a": 0.052,
+        "share_b": 0.0,
+        "ratio": 393.82
+      },
+      {
+        "token": "scanning",
+        "share_a": 0.052,
+        "share_b": 0.0,
+        "ratio": 393.82
+      },
+      {
+        "token": "matched",
+        "share_a": 0.052,
+        "share_b": 0.0,
+        "ratio": 131.27
+      },
+      {
+        "token": "producing",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "nasdaq",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "fotis",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "usages",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "targeting",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "ibd",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "gannett",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "yhoo",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "ringing",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "twx",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "goog",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "greko's",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "gci",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      },
+      {
+        "token": "e-marketing",
+        "share_a": 0.099,
+        "share_b": 0.001,
+        "ratio": 105.77
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "ips",
+        "share_a": 0.051,
+        "share_b": 0.002,
+        "ratio": 27.08
+      },
+      {
+        "token": "noc's",
+        "share_a": 0.051,
+        "share_b": 0.002,
+        "ratio": 23.9
+      },
+      {
+        "token": "filtered",
+        "share_a": 0.051,
+        "share_b": 0.003,
+        "ratio": 15.83
+      },
+      {
+        "token": "tcpview",
+        "share_a": 0.051,
+        "share_b": 0.003,
+        "ratio": 15.43
+      },
+      {
+        "token": "procexp",
+        "share_a": 0.051,
+        "share_b": 0.003,
+        "ratio": 15.43
+      },
+      {
+        "token": "sysinternals",
+        "share_a": 0.051,
+        "share_b": 0.003,
+        "ratio": 15.05
+      },
+      {
+        "token": "atkins",
+        "share_a": 0.051,
+        "share_b": 0.004,
+        "ratio": 14.34
+      },
+      {
+        "token": "rogue",
+        "share_a": 0.051,
+        "share_b": 0.004,
+        "ratio": 13.39
+      },
+      {
+        "token": "editorial",
+        "share_a": 0.051,
+        "share_b": 0.004,
+        "ratio": 12.07
+      },
+      {
+        "token": "lsdepts",
+        "share_a": 0.051,
+        "share_b": 0.004,
+        "ratio": 12.07
+      },
+      {
+        "token": "csf",
+        "share_a": 0.052,
+        "share_b": 0.004,
+        "ratio": 11.76
+      },
+      {
+        "token": "csf-bounces",
+        "share_a": 0.051,
+        "share_b": 0.004,
+        "ratio": 11.61
+      },
+      {
+        "token": "aes256-sha",
+        "share_a": 0.051,
+        "share_b": 0.004,
+        "ratio": 11.61
+      },
+      {
+        "token": "oit",
+        "share_a": 0.052,
+        "share_b": 0.005,
+        "ratio": 11.55
+      },
+      {
+        "token": "irc",
+        "share_a": 0.056,
+        "share_b": 0.005,
+        "ratio": 11.44
+      },
+      {
+        "token": "list-post",
+        "share_a": 0.051,
+        "share_b": 0.004,
+        "ratio": 11.39
+      },
+      {
+        "token": "list-subscribe",
+        "share_a": 0.051,
+        "share_b": 0.004,
+        "ratio": 11.39
+      },
+      {
+        "token": "rebuilt",
+        "share_a": 0.051,
+        "share_b": 0.004,
+        "ratio": 11.39
+      },
+      {
+        "token": "list-archive",
+        "share_a": 0.051,
+        "share_b": 0.004,
+        "ratio": 11.39
+      },
+      {
+        "token": "ucsb",
+        "share_a": 0.052,
+        "share_b": 0.005,
+        "ratio": 11.34
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:052869",
+      "primary:086987",
+      "primary:044992",
+      "primary:173636",
+      "primary:080991",
+      "primary:145723",
+      "primary:143065",
+      "primary:006031",
+      "primary:019350",
+      "primary:172278",
+      "primary:033014",
+      "primary:156328",
+      "primary:009091",
+      "primary:176328",
+      "primary:076023",
+      "primary:161198",
+      "primary:122099",
+      "primary:090853",
+      "primary:160104",
+      "primary:121268",
+      "primary:024565",
+      "primary:119133",
+      "primary:020947",
+      "primary:064709",
+      "primary:171201"
+    ],
+    "top_fp_doc_ids": [
+      "primary:143547",
+      "primary:163929",
+      "primary:113500",
+      "primary:118028",
+      "primary:202546",
+      "primary:113156",
+      "primary:107132",
+      "primary:126805",
+      "primary:070866",
+      "primary:062395",
+      "primary:071954",
+      "primary:069141",
+      "primary:158244",
+      "primary:040048",
+      "primary:170605",
+      "primary:106445",
+      "primary:084432",
+      "primary:124491",
+      "primary:196901",
+      "primary:101560",
+      "primary:078484",
+      "primary:049454",
+      "primary:002284",
+      "primary:068707",
+      "primary:136688"
+    ],
+    "threeway": null
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-ce/confusion_matrix.png",
+    "roc_curve": "reports/eval_ft-ce/roc_curve.png",
+    "pr_curve": "reports/eval_ft-ce/pr_curve.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-ce/metrics_test3.json b/presentation/reports/eval_ft-ce/metrics_test3.json
new file mode 100644
index 0000000000000000000000000000000000000000..320cc86aae236b1b8ca2ddd70fe58443334078b8
--- /dev/null
+++ b/presentation/reports/eval_ft-ce/metrics_test3.json
@@ -0,0 +1,1729 @@
+{
+  "timestamp": "2026-10-09 04:28",
+  "run_name": "ft-ce",
+  "split": "test3",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout3.parquet",
+  "split_title": "three-way holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-ce/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-ce",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "cross_entropy",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 3,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 0,
+    "best_step": 5000,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.992930086410055,
+    "val_metrics": {
+      "accuracy": 0.9923449358806595,
+      "precision": 0.9938308939155679,
+      "recall": 0.992030910408114,
+      "f1": 0.992930086410055,
+      "roc_auc": 0.9995503578823617,
+      "pr_auc": 0.9996107014519675,
+      "tn": 6951,
+      "fp": 51,
+      "fn": 66,
+      "tp": 8216,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.31750011444091797,
+      "f1_at_optimal_threshold": 0.9931808581256412,
+      "f1_unwanted": 0.992930086410055,
+      "n_3way": 8105,
+      "accuracy_3way": 0.9922270203578039,
+      "macro_f1_3way": 0.9781799660145108,
+      "precision_legitimate": 0.9988504095416008,
+      "recall_legitimate": 0.9927163667523564,
+      "f1_legitimate": 0.995773941694721,
+      "precision_phishing": 0.9390581717451524,
+      "recall_phishing": 0.9970588235294118,
+      "f1_phishing": 0.9671897289586305,
+      "precision_spam": 0.9579617834394905,
+      "recall_spam": 0.9855832241153342,
+      "f1_spam": 0.9715762273901809,
+      "cm3_ll": 6951,
+      "cm3_lp": 18,
+      "cm3_ls": 33,
+      "cm3_pl": 1,
+      "cm3_pp": 339,
+      "cm3_ps": 0,
+      "cm3_sl": 7,
+      "cm3_sp": 4,
+      "cm3_ss": 752,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 340,
+      "pvs_phish_as_spam": 0,
+      "pvs_spam_as_phish": 4,
+      "pvs_spam_as_spam": 759,
+      "val_loss": 0.027098098741997853,
+      "n": 15284,
+      "eval_seconds": 47.7
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9923449358806595,
+        "precision": 0.9938308939155679,
+        "recall": 0.992030910408114,
+        "f1": 0.992930086410055,
+        "roc_auc": 0.9995503578823617,
+        "pr_auc": 0.9996107014519675,
+        "tn": 6951,
+        "fp": 51,
+        "fn": 66,
+        "tp": 8216,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.31750011444091797,
+        "f1_at_optimal_threshold": 0.9931808581256412,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.9922270203578039,
+        "macro_f1": 0.9781799660145108,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9988504095416008,
+            "recall": 0.9927163667523564,
+            "f1": 0.995773941694721
+          },
+          "Phishing": {
+            "precision": 0.9390581717451524,
+            "recall": 0.9970588235294118,
+            "f1": 0.9671897289586305
+          },
+          "Spam": {
+            "precision": 0.9579617834394905,
+            "recall": 0.9855832241153342,
+            "f1": 0.9715762273901809
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6951,
+              18,
+              33
+            ],
+            [
+              1,
+              339,
+              0
+            ],
+            [
+              7,
+              4,
+              752
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              340,
+              0
+            ],
+            [
+              4,
+              759
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.31750011444091797,
+      "val_f1_at_optimal": 0.9931808581256412
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 3220,
+    "positive": 1221,
+    "negative": 1999,
+    "positive_rate": 0.3792,
+    "label3": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "by_source": {
+      "Assassin": {
+        "rows": 236,
+        "positive": 72,
+        "label3": {
+          "legit": 164,
+          "phish": 0,
+          "spam": 72,
+          "unwanted": 0
+        }
+      },
+      "CEAS-08": {
+        "rows": 736,
+        "positive": 0,
+        "label3": {
+          "legit": 736,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Enron": {
+        "rows": 628,
+        "positive": 628,
+        "label3": {
+          "legit": 0,
+          "phish": 0,
+          "spam": 628,
+          "unwanted": 0
+        }
+      },
+      "Nazario": {
+        "rows": 222,
+        "positive": 222,
+        "label3": {
+          "legit": 0,
+          "phish": 222,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Nigerian_Fraud": {
+        "rows": 299,
+        "positive": 299,
+        "label3": {
+          "legit": 0,
+          "phish": 299,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "TREC-07": {
+        "rows": 1099,
+        "positive": 0,
+        "label3": {
+          "legit": 1099,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      }
+    },
+    "mean_tokens_full": 835.2,
+    "pct_truncated": 34.01
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.989751552795031,
+    "precision": 0.9876847290640394,
+    "recall": 0.9852579852579852,
+    "f1": 0.986469864698647,
+    "specificity": 0.992496248124062,
+    "false_positive_rate": 0.007503751875937969,
+    "false_negative_rate": 0.014742014742014743,
+    "confusion_matrix": {
+      "tn": 1984,
+      "fp": 15,
+      "fn": 18,
+      "tp": 1203
+    },
+    "n": 3220,
+    "n_positive": 1221,
+    "n_negative": 1999
+  },
+  "threshold_free": {
+    "roc_auc": 0.9991504761389705,
+    "pr_auc": 0.9985052105989761,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.9868959868959869,
+      "fpr": 0.008504252126063032,
+      "threshold": 0.3451557755470276,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.868959868959869,
+      "fpr": 0.0005002501250625312,
+      "threshold": 0.9972003102302551,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.41115501523017883,
+      "f1": 0.9868852459016394,
+      "precision": 0.9876948318293683,
+      "recall": 0.9860769860769861
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 1203,
+        "predicted_phishing": 513,
+        "predicted_spam": 690
+      },
+      "false_positives_flagged": {
+        "n": 15,
+        "predicted_phishing": 4,
+        "predicted_spam": 11
+      },
+      "missed_positives": {
+        "n": 18,
+        "predicted_phishing": 7,
+        "predicted_spam": 11
+      },
+      "all_flagged": {
+        "n": 1218,
+        "predicted_phishing": 517,
+        "predicted_spam": 701
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 0,
+        "predicted_phishing": 0,
+        "predicted_spam": 0
+      }
+    },
+    "rows_with_full_label": 3220,
+    "rows_unwanted_only": 0,
+    "evaluated": true,
+    "reason": null,
+    "confusion_matrix": {
+      "labels": [
+        "Legitimate",
+        "Phishing",
+        "Spam"
+      ],
+      "matrix": [
+        [
+          1984,
+          4,
+          11
+        ],
+        [
+          9,
+          507,
+          5
+        ],
+        [
+          9,
+          6,
+          685
+        ]
+      ],
+      "cells": {
+        "true_legitimate": {
+          "pred_legitimate": 1984,
+          "pred_phishing": 4,
+          "pred_spam": 11
+        },
+        "true_phishing": {
+          "pred_legitimate": 9,
+          "pred_phishing": 507,
+          "pred_spam": 5
+        },
+        "true_spam": {
+          "pred_legitimate": 9,
+          "pred_phishing": 6,
+          "pred_spam": 685
+        }
+      }
+    },
+    "accuracy": 0.986335403726708,
+    "macro_f1": 0.9821678741985428,
+    "macro_f1_present_classes": 0.9821678741985428,
+    "classes_present": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "per_class": {
+      "Legitimate": {
+        "precision": 0.991008991008991,
+        "recall": 0.992496248124062,
+        "f1": 0.9917520619845038,
+        "support": 1999
+      },
+      "Phishing": {
+        "precision": 0.9806576402321083,
+        "recall": 0.9731285988483686,
+        "f1": 0.976878612716763,
+        "support": 521
+      },
+      "Spam": {
+        "precision": 0.9771754636233951,
+        "recall": 0.9785714285714285,
+        "f1": 0.9778729478943612,
+        "support": 700
+      }
+    },
+    "argmax_rule": {
+      "doc": "plain argmax over the three probabilities (secondary; can disagree with the binary verdict near the threshold)",
+      "accuracy": 0.9866459627329193,
+      "macro_f1": 0.9825658607822209,
+      "per_class": {
+        "Legitimate": {
+          "precision": 0.9910134797803295,
+          "recall": 0.9929964982491246,
+          "f1": 0.9920039980009995,
+          "support": 1999
+        },
+        "Phishing": {
+          "precision": 0.9825581395348837,
+          "recall": 0.9731285988483686,
+          "f1": 0.9778206364513018,
+          "support": 521
+        },
+        "Spam": {
+          "precision": 0.9771754636233951,
+          "recall": 0.9785714285714285,
+          "f1": 0.9778729478943612,
+          "support": 700
+        }
+      },
+      "rows_where_rules_differ": 1
+    },
+    "phish_vs_spam_on_true_unwanted": {
+      "n_true_unwanted": 1221,
+      "kind_only": {
+        "doc": "given the row is unwanted, is argmax(P(phishing), P(spam)) right? (independent of the roll-up threshold)",
+        "labels": [
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            514,
+            7
+          ],
+          [
+            6,
+            694
+          ]
+        ],
+        "cells": {
+          "true_phishing": {
+            "pred_phishing": 514,
+            "pred_spam": 7
+          },
+          "true_spam": {
+            "pred_phishing": 6,
+            "pred_spam": 694
+          }
+        },
+        "accuracy": 0.9893529893529893,
+        "phishing_as_spam": 7,
+        "spam_as_phishing": 6
+      },
+      "rollup_rule": {
+        "doc": "rows = true Phishing / Spam; columns = predicted Legitimate (missed) / Phishing / Spam under the roll-up rule",
+        "rows": [
+          "Phishing",
+          "Spam"
+        ],
+        "columns": [
+          "Legitimate",
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            9,
+            507,
+            5
+          ],
+          [
+            9,
+            6,
+            685
+          ]
+        ],
+        "missed_as_legitimate": {
+          "phishing": 9,
+          "spam": 9
+        }
+      }
+    },
+    "per_source": {
+      "Assassin": {
+        "n": 236,
+        "counts": {
+          "Legitimate": 164,
+          "Phishing": 0,
+          "Spam": 72
+        },
+        "accuracy": 0.9449152542372882,
+        "macro_f1": 0.6309157509157509,
+        "macro_f1_present_classes": 0.9463736263736264,
+        "per_class_recall": {
+          "Legitimate": 0.9695121951219512,
+          "Spam": 0.8888888888888888
+        }
+      },
+      "CEAS-08": {
+        "n": 736,
+        "counts": {
+          "Legitimate": 736,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9904891304347826,
+        "macro_f1": 0.331740614334471,
+        "macro_f1_present_classes": 0.995221843003413,
+        "per_class_recall": {
+          "Legitimate": 0.9904891304347826
+        }
+      },
+      "Enron": {
+        "n": 628,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 0,
+          "Spam": 628
+        },
+        "accuracy": 0.9888535031847133,
+        "macro_f1": 0.33146517213771015,
+        "macro_f1_present_classes": 0.9943955164131305,
+        "per_class_recall": {
+          "Spam": 0.9888535031847133
+        }
+      },
+      "Nazario": {
+        "n": 222,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 222,
+          "Spam": 0
+        },
+        "accuracy": 0.9504504504504504,
+        "macro_f1": 0.3248652809853734,
+        "macro_f1_present_classes": 0.9745958429561201,
+        "per_class_recall": {
+          "Phishing": 0.9504504504504504
+        }
+      },
+      "Nigerian_Fraud": {
+        "n": 299,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 299,
+          "Spam": 0
+        },
+        "accuracy": 0.9899665551839465,
+        "macro_f1": 0.33165266106442576,
+        "macro_f1_present_classes": 0.9949579831932773,
+        "per_class_recall": {
+          "Phishing": 0.9899665551839465
+        }
+      },
+      "TREC-07": {
+        "n": 1099,
+        "counts": {
+          "Legitimate": 1099,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.997270245677889,
+        "macro_f1": 0.3328777524677297,
+        "macro_f1_present_classes": 0.9986332574031891,
+        "per_class_recall": {
+          "Legitimate": 0.997270245677889
+        }
+      }
+    }
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 139,
+      "bank account": 91,
+      "urgently": 25,
+      "verify your account": 15,
+      "confirm your account": 9,
+      "update your account": 9,
+      "within 24 hours": 8,
+      "security alert": 8,
+      "paypal account": 7,
+      "password will expire": 7,
+      "click here to verify": 6,
+      "unusual activity": 4,
+      "wire transfer": 3,
+      "update your information": 3,
+      "gift card": 3,
+      "verify your identity": 2,
+      "unusual sign-in": 2,
+      "account will be suspended": 2,
+      "update your payment": 2,
+      "failure to verify": 2,
+      "within 48 hours": 2,
+      "confirm your identity": 2,
+      "enter your password": 1,
+      "update your billing": 1,
+      "reset your password": 1,
+      "suspicious activity": 1,
+      "security notice": 1,
+      "log in to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 15,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "update your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 2,
+        "negative": 1
+      },
+      "update your billing": {
+        "positive": 1,
+        "negative": 0
+      },
+      "update your payment": {
+        "positive": 2,
+        "negative": 0
+      },
+      "click here to verify": {
+        "positive": 6,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual sign-in": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "suspicious activity": {
+        "positive": 1,
+        "negative": 0
+      },
+      "paypal account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "password will expire": {
+        "positive": 7,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 1,
+        "negative": 0
+      },
+      "wire transfer": {
+        "positive": 2,
+        "negative": 1
+      },
+      "gift card": {
+        "positive": 3,
+        "negative": 0
+      },
+      "urgent": {
+        "positive": 133,
+        "negative": 6
+      },
+      "urgently": {
+        "positive": 25,
+        "negative": 0
+      },
+      "security alert": {
+        "positive": 8,
+        "negative": 0
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 91,
+        "negative": 0
+      },
+      "failure to verify": {
+        "positive": 2,
+        "negative": 0
+      },
+      "within 24 hours": {
+        "positive": 8,
+        "negative": 0
+      },
+      "within 48 hours": {
+        "positive": 2,
+        "negative": 0
+      }
+    },
+    "label_purity": 0.9745,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 7,
+      "phish": 236,
+      "spam": 32,
+      "unwanted": 0
+    },
+    "threeway_accuracy": 0.9745454545454545,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 0.9927272727272727,
+      "precision": 1.0,
+      "recall": 0.9925373134328358,
+      "f1": 0.9962546816479401,
+      "specificity": 1.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.007462686567164179,
+      "confusion_matrix": {
+        "tn": 7,
+        "fp": 0,
+        "fn": 2,
+        "tp": 266
+      },
+      "n": 275,
+      "n_positive": 268,
+      "n_negative": 7,
+      "roc_auc": 1.0
+    }
+  },
+  "per_source": {
+    "Assassin": {
+      "threshold": 0.5,
+      "accuracy": 0.9703389830508474,
+      "precision": 0.9333333333333333,
+      "recall": 0.9722222222222222,
+      "f1": 0.9523809523809524,
+      "specificity": 0.9695121951219512,
+      "false_positive_rate": 0.03048780487804878,
+      "false_negative_rate": 0.027777777777777776,
+      "confusion_matrix": {
+        "tn": 159,
+        "fp": 5,
+        "fn": 2,
+        "tp": 70
+      },
+      "n": 236,
+      "n_positive": 72,
+      "n_negative": 164,
+      "roc_auc": 0.9962737127371274,
+      "label3_counts": {
+        "legit": 164,
+        "phish": 0,
+        "spam": 72,
+        "unwanted": 0
+      }
+    },
+    "CEAS-08": {
+      "threshold": 0.5,
+      "accuracy": 0.9904891304347826,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9904891304347826,
+      "false_positive_rate": 0.009510869565217392,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 729,
+        "fp": 7,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 736,
+      "n_positive": 0,
+      "n_negative": 736,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 736,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Enron": {
+      "threshold": 0.5,
+      "accuracy": 0.9888535031847133,
+      "precision": 1.0,
+      "recall": 0.9888535031847133,
+      "f1": 0.9943955164131305,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.011146496815286623,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 7,
+        "tp": 621
+      },
+      "n": 628,
+      "n_positive": 628,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 0,
+        "spam": 628,
+        "unwanted": 0
+      }
+    },
+    "Nazario": {
+      "threshold": 0.5,
+      "accuracy": 0.963963963963964,
+      "precision": 1.0,
+      "recall": 0.963963963963964,
+      "f1": 0.9816513761467891,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.036036036036036036,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 8,
+        "tp": 214
+      },
+      "n": 222,
+      "n_positive": 222,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 222,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Nigerian_Fraud": {
+      "threshold": 0.5,
+      "accuracy": 0.9966555183946488,
+      "precision": 1.0,
+      "recall": 0.9966555183946488,
+      "f1": 0.998324958123953,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.0033444816053511705,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 1,
+        "tp": 298
+      },
+      "n": 299,
+      "n_positive": 299,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 299,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "TREC-07": {
+      "threshold": 0.5,
+      "accuracy": 0.997270245677889,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.997270245677889,
+      "false_positive_rate": 0.00272975432211101,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 1096,
+        "fp": 3,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 1099,
+      "n_positive": 0,
+      "n_negative": 1099,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 1099,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 1203,
+        "mean_tokens_full": 618.5,
+        "mean_tokens_fed": 280.7,
+        "median_tokens_full": 244.0,
+        "pct_truncated": 26.7
+      },
+      "tn": {
+        "n": 1984,
+        "mean_tokens_full": 731.9,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.4
+      },
+      "fp": {
+        "n": 15,
+        "mean_tokens_full": 1771.5,
+        "mean_tokens_fed": 357.9,
+        "median_tokens_full": 384.0,
+        "pct_truncated": 46.7
+      },
+      "fn": {
+        "n": 18,
+        "mean_tokens_full": 25929.3,
+        "mean_tokens_fed": 223.4,
+        "median_tokens_full": 121.0,
+        "pct_truncated": 27.8
+      },
+      "correct": {
+        "n": 3187,
+        "mean_tokens_full": 689.1,
+        "mean_tokens_fed": 329.2,
+        "median_tokens_full": 339.0,
+        "pct_truncated": 34.0
+      },
+      "wrong": {
+        "n": 33,
+        "mean_tokens_full": 14948.5,
+        "mean_tokens_fed": 284.5,
+        "median_tokens_full": 339.0,
+        "pct_truncated": 36.4
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "unwanted": {
+        "n": 1221,
+        "mean_tokens_full": 991.6,
+        "mean_tokens_fed": 279.9,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.7
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "phish": {
+        "n": 521,
+        "mean_tokens_full": 1349.4,
+        "mean_tokens_fed": 355.1,
+        "median_tokens_full": 412.0,
+        "pct_truncated": 39.9
+      },
+      "spam": {
+        "n": 700,
+        "mean_tokens_full": 725.3,
+        "mean_tokens_fed": 223.9,
+        "median_tokens_full": 159.0,
+        "pct_truncated": 16.9
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 129,
+        "n_positive": 99,
+        "positive_rate": 0.7674418604651163,
+        "recall": 0.9393939393939394,
+        "fpr": 0.03333333333333333,
+        "accuracy": 0.9457364341085271
+      },
+      {
+        "bucket": "65-128",
+        "n": 427,
+        "n_positive": 252,
+        "positive_rate": 0.5901639344262295,
+        "recall": 0.9880952380952381,
+        "fpr": 0.011428571428571429,
+        "accuracy": 0.9882903981264637
+      },
+      {
+        "bucket": "129-256",
+        "n": 699,
+        "n_positive": 276,
+        "positive_rate": 0.3948497854077253,
+        "recall": 0.9927536231884058,
+        "fpr": 0.002364066193853428,
+        "accuracy": 0.9957081545064378
+      },
+      {
+        "bucket": "257-512",
+        "n": 870,
+        "n_positive": 268,
+        "positive_rate": 0.3080459770114943,
+        "recall": 0.9925373134328358,
+        "fpr": 0.006644518272425249,
+        "accuracy": 0.993103448275862
+      },
+      {
+        "bucket": ">512",
+        "n": 1095,
+        "n_positive": 326,
+        "positive_rate": 0.29771689497716897,
+        "recall": 0.9846625766871165,
+        "fpr": 0.009102730819245773,
+        "accuracy": 0.989041095890411
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 3220,
+    "seconds": 13.37,
+    "rows_per_s": 240.9,
+    "padded_tokens": 1079936,
+    "real_tokens": 1058490,
+    "padded_tokens_per_s": 80797,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 9.95,
+      "p50_ms": 8.72,
+      "p95_ms": 13.58,
+      "max_ms": 19.1,
+      "avg_s": 0.0099,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.134,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 514.07,
+      "p50_ms": 526.19,
+      "p95_ms": 724.38,
+      "max_ms": 783.31,
+      "avg_s": 0.5141,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 6.947,
+      "threads": 16,
+      "estimated_full_run_s": 51.3
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-ce/best",
+    "run_name": "ft-ce",
+    "params": 337444867,
+    "timestamp": "2026-10-09 04:28",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-ce/error_analysis_test3.md",
+    "n_fn": 18,
+    "n_fp": 15,
+    "composition": {
+      "fp": {
+        "n": 15,
+        "mailing_list_or_digest": 0.4,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.467,
+        "lure_phrase": 0.0
+      },
+      "tn": {
+        "n": 1984,
+        "mailing_list_or_digest": 0.564,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.384,
+        "lure_phrase": 0.004
+      },
+      "fn": {
+        "n": 18,
+        "mailing_list_or_digest": 0.222,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.056,
+        "truncated_gt512": 0.278,
+        "lure_phrase": 0.111
+      },
+      "tp": {
+        "n": 1203,
+        "mailing_list_or_digest": 0.085,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.007,
+        "truncated_gt512": 0.267,
+        "lure_phrase": 0.221
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "music",
+        "share_a": 0.167,
+        "share_b": 0.005,
+        "ratio": 35.03
+      },
+      {
+        "token": "systems",
+        "share_a": 0.167,
+        "share_b": 0.006,
+        "ratio": 30.36
+      },
+      {
+        "token": "social",
+        "share_a": 0.167,
+        "share_b": 0.012,
+        "ratio": 14.69
+      },
+      {
+        "token": "central",
+        "share_a": 0.167,
+        "share_b": 0.016,
+        "ratio": 11.68
+      },
+      {
+        "token": "computer",
+        "share_a": 0.167,
+        "share_b": 0.031,
+        "ratio": 6.07
+      },
+      {
+        "token": "american",
+        "share_a": 0.167,
+        "share_b": 0.032,
+        "ratio": 5.91
+      },
+      {
+        "token": "add",
+        "share_a": 0.167,
+        "share_b": 0.034,
+        "ratio": 5.49
+      },
+      {
+        "token": "change",
+        "share_a": 0.222,
+        "share_b": 0.051,
+        "ratio": 4.76
+      },
+      {
+        "token": "info",
+        "share_a": 0.278,
+        "share_b": 0.062,
+        "ratio": 4.74
+      },
+      {
+        "token": "remove",
+        "share_a": 0.167,
+        "share_b": 0.054,
+        "ratio": 3.48
+      },
+      {
+        "token": "thank",
+        "share_a": 0.278,
+        "share_b": 0.087,
+        "ratio": 3.39
+      },
+      {
+        "token": "much",
+        "share_a": 0.222,
+        "share_b": 0.072,
+        "ratio": 3.35
+      },
+      {
+        "token": "available",
+        "share_a": 0.167,
+        "share_b": 0.061,
+        "ratio": 3.1
+      },
+      {
+        "token": "off",
+        "share_a": 0.167,
+        "share_b": 0.062,
+        "ratio": 3.02
+      },
+      {
+        "token": "subject",
+        "share_a": 0.167,
+        "share_b": 0.067,
+        "ratio": 2.83
+      },
+      {
+        "token": "list",
+        "share_a": 0.222,
+        "share_b": 0.086,
+        "ratio": 2.8
+      },
+      {
+        "token": "call",
+        "share_a": 0.167,
+        "share_b": 0.067,
+        "ratio": 2.79
+      },
+      {
+        "token": "view",
+        "share_a": 0.167,
+        "share_b": 0.069,
+        "ratio": 2.73
+      },
+      {
+        "token": "hope",
+        "share_a": 0.167,
+        "share_b": 0.071,
+        "ratio": 2.63
+      },
+      {
+        "token": "might",
+        "share_a": 0.167,
+        "share_b": 0.071,
+        "ratio": 2.63
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "outlets",
+        "share_a": 0.2,
+        "share_b": 0.001,
+        "ratio": 179.25
+      },
+      {
+        "token": "czzz-unspun",
+        "share_a": 0.2,
+        "share_b": 0.001,
+        "ratio": 179.25
+      },
+      {
+        "token": "zzz-unspun",
+        "share_a": 0.2,
+        "share_b": 0.001,
+        "ratio": 179.25
+      },
+      {
+        "token": "full-color",
+        "share_a": 0.333,
+        "share_b": 0.002,
+        "ratio": 156.48
+      },
+      {
+        "token": "spins",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "guterman",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "savviest",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "erikvanderkolk",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "eletra",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "unspun",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "mld",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "muehlbauer",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "authoritatively",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "lori",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "jen",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "duffy",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "vanderkolk",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "chtm",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "msnbc",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      },
+      {
+        "token": "imakenews",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 128.03
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:191966",
+      "aura:Nazario:000097",
+      "primary:148887",
+      "aura:Nazario:001540",
+      "aura:Nazario:000551",
+      "aura:Nazario:000643",
+      "primary:102650",
+      "primary:007176",
+      "aura:Nazario:000299",
+      "primary:067343",
+      "primary:173930",
+      "primary:181246",
+      "aura:Nazario:000300",
+      "aura:Nazario:000533",
+      "aura:Nigerian_Fraud:003049",
+      "aura:Nazario:000189",
+      "primary:025024",
+      "primary:193295"
+    ],
+    "top_fp_doc_ids": [
+      "primary:144048",
+      "primary:199386",
+      "primary:033114",
+      "primary:165228",
+      "primary:156887",
+      "primary:153841",
+      "primary:041713",
+      "primary:152763",
+      "primary:055965",
+      "primary:117398",
+      "primary:103927",
+      "primary:078627",
+      "primary:109226",
+      "primary:153654",
+      "primary:116110"
+    ],
+    "threeway": {
+      "n_phish_as_spam": 5,
+      "n_spam_as_phish": 6,
+      "top_phish_as_spam_doc_ids": [
+        "aura:Nigerian_Fraud:001698",
+        "aura:Nazario:000532",
+        "aura:Nazario:000105",
+        "aura:Nazario:000104",
+        "aura:Nigerian_Fraud:000337"
+      ],
+      "top_spam_as_phish_doc_ids": [
+        "primary:077835",
+        "primary:028382",
+        "primary:087246",
+        "primary:197304",
+        "primary:129677",
+        "primary:198712"
+      ],
+      "phish_as_spam_tokens": [
+        {
+          "token": "complete",
+          "share_a": 0.6,
+          "share_b": 0.049,
+          "ratio": 12.66
+        },
+        {
+          "token": "www",
+          "share_a": 0.6,
+          "share_b": 0.15,
+          "ratio": 4.22
+        },
+        {
+          "token": "http",
+          "share_a": 0.6,
+          "share_b": 0.195,
+          "ratio": 3.25
+        },
+        {
+          "token": "com",
+          "share_a": 0.6,
+          "share_b": 0.339,
+          "ratio": 1.87
+        },
+        {
+          "token": "here",
+          "share_a": 0.6,
+          "share_b": 0.414,
+          "ratio": 1.53
+        },
+        {
+          "token": "are",
+          "share_a": 0.6,
+          "share_b": 0.57,
+          "ratio": 1.12
+        },
+        {
+          "token": "will",
+          "share_a": 0.6,
+          "share_b": 0.722,
+          "ratio": 0.88
+        },
+        {
+          "token": "you",
+          "share_a": 0.6,
+          "share_b": 0.901,
+          "ratio": 0.71
+        },
+        {
+          "token": "your",
+          "share_a": 0.6,
+          "share_b": 0.929,
+          "ratio": 0.68
+        }
+      ],
+      "spam_as_phish_tokens": [
+        {
+          "token": "average",
+          "share_a": 0.5,
+          "share_b": 0.009,
+          "ratio": 56.79
+        },
+        {
+          "token": "share",
+          "share_a": 0.5,
+          "share_b": 0.023,
+          "ratio": 22.37
+        },
+        {
+          "token": "assistance",
+          "share_a": 0.5,
+          "share_b": 0.026,
+          "ratio": 19.95
+        },
+        {
+          "token": "upon",
+          "share_a": 0.5,
+          "share_b": 0.031,
+          "ratio": 17.17
+        },
+        {
+          "token": "private",
+          "share_a": 0.667,
+          "share_b": 0.041,
+          "ratio": 16.65
+        },
+        {
+          "token": "call",
+          "share_a": 0.5,
+          "share_b": 0.045,
+          "ratio": 11.72
+        },
+        {
+          "token": "immediately",
+          "share_a": 0.5,
+          "share_b": 0.048,
+          "ratio": 11.02
+        },
+        {
+          "token": "account",
+          "share_a": 0.5,
+          "share_b": 0.05,
+          "ratio": 10.7
+        },
+        {
+          "token": "got",
+          "share_a": 0.5,
+          "share_b": 0.051,
+          "ratio": 10.4
+        },
+        {
+          "token": "under",
+          "share_a": 0.5,
+          "share_b": 0.051,
+          "ratio": 10.4
+        },
+        {
+          "token": "company",
+          "share_a": 0.667,
+          "share_b": 0.067,
+          "ratio": 10.21
+        },
+        {
+          "token": "way",
+          "share_a": 0.5,
+          "share_b": 0.058,
+          "ratio": 9.11
+        },
+        {
+          "token": "next",
+          "share_a": 0.5,
+          "share_b": 0.06,
+          "ratio": 8.89
+        },
+        {
+          "token": "help",
+          "share_a": 0.5,
+          "share_b": 0.064,
+          "ratio": 8.29
+        },
+        {
+          "token": "could",
+          "share_a": 0.5,
+          "share_b": 0.066,
+          "ratio": 8.11
+        },
+        {
+          "token": "people",
+          "share_a": 0.5,
+          "share_b": 0.069,
+          "ratio": 7.77
+        },
+        {
+          "token": "last",
+          "share_a": 0.5,
+          "share_b": 0.069,
+          "ratio": 7.77
+        },
+        {
+          "token": "year",
+          "share_a": 0.5,
+          "share_b": 0.07,
+          "ratio": 7.61
+        },
+        {
+          "token": "take",
+          "share_a": 0.667,
+          "share_b": 0.101,
+          "ratio": 6.83
+        },
+        {
+          "token": "they",
+          "share_a": 0.667,
+          "share_b": 0.102,
+          "ratio": 6.73
+        }
+      ]
+    }
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-ce/confusion_matrix_test3.png",
+    "confusion_matrix3": "reports/eval_ft-ce/confusion_matrix3_test3.png",
+    "roc_curve": "reports/eval_ft-ce/roc_curve_test3.png",
+    "pr_curve": "reports/eval_ft-ce/pr_curve_test3.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-focal/metrics.json b/presentation/reports/eval_ft-focal/metrics.json
new file mode 100644
index 0000000000000000000000000000000000000000..f34db6e89e383a6455b0ffa5f26c34f1e671b914
--- /dev/null
+++ b/presentation/reports/eval_ft-focal/metrics.json
@@ -0,0 +1,1109 @@
+{
+  "timestamp": "2026-10-09 07:22",
+  "run_name": "ft-focal",
+  "split": "test",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet",
+  "split_title": "TREC-06 binary holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-focal",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "focal",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 1,
+    "best_step": 8750,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9951084002657166,
+    "val_metrics": {
+      "accuracy": 0.9947003402250719,
+      "precision": 0.9954089646007007,
+      "recall": 0.9948080173871046,
+      "f1": 0.9951084002657166,
+      "roc_auc": 0.9998186946414248,
+      "pr_auc": 0.9998464119156345,
+      "tn": 6964,
+      "fp": 38,
+      "fn": 43,
+      "tp": 8239,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.42074909806251526,
+      "f1_at_optimal_threshold": 0.9952995058454863,
+      "f1_unwanted": 0.9951084002657166,
+      "n_3way": 8105,
+      "accuracy_3way": 0.994694632942628,
+      "macro_f1_3way": 0.9868751212108838,
+      "precision_legitimate": 0.9988533753762362,
+      "recall_legitimate": 0.99528706083976,
+      "f1_legitimate": 0.9970670291151013,
+      "precision_phishing": 0.9853372434017595,
+      "recall_phishing": 0.9882352941176471,
+      "f1_phishing": 0.986784140969163,
+      "precision_spam": 0.9618805590851334,
+      "recall_spam": 0.9921363040629095,
+      "f1_spam": 0.9767741935483871,
+      "cm3_ll": 6969,
+      "cm3_lp": 4,
+      "cm3_ls": 29,
+      "cm3_pl": 3,
+      "cm3_pp": 336,
+      "cm3_ps": 1,
+      "cm3_sl": 5,
+      "cm3_sp": 1,
+      "cm3_ss": 757,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 339,
+      "pvs_phish_as_spam": 1,
+      "pvs_spam_as_phish": 1,
+      "pvs_spam_as_spam": 762,
+      "val_loss": 0.030296484990695233,
+      "n": 15284,
+      "eval_seconds": 48.0
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9947003402250719,
+        "precision": 0.9954089646007007,
+        "recall": 0.9948080173871046,
+        "f1": 0.9951084002657166,
+        "roc_auc": 0.9998186946414248,
+        "pr_auc": 0.9998464119156345,
+        "tn": 6964,
+        "fp": 38,
+        "fn": 43,
+        "tp": 8239,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.42074909806251526,
+        "f1_at_optimal_threshold": 0.9952995058454863,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.994694632942628,
+        "macro_f1": 0.9868751212108838,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9988533753762362,
+            "recall": 0.99528706083976,
+            "f1": 0.9970670291151013
+          },
+          "Phishing": {
+            "precision": 0.9853372434017595,
+            "recall": 0.9882352941176471,
+            "f1": 0.986784140969163
+          },
+          "Spam": {
+            "precision": 0.9618805590851334,
+            "recall": 0.9921363040629095,
+            "f1": 0.9767741935483871
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6969,
+              4,
+              29
+            ],
+            [
+              3,
+              336,
+              1
+            ],
+            [
+              5,
+              1,
+              757
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              339,
+              1
+            ],
+            [
+              1,
+              762
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.42074909806251526,
+      "val_f1_at_optimal": 0.9952995058454863
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 16302,
+    "positive": 3894,
+    "negative": 12408,
+    "positive_rate": 0.2389,
+    "label3": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "by_source": {
+      "TREC-06": {
+        "rows": 16302,
+        "positive": 3894,
+        "label3": {
+          "legit": 12408,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 3894
+        }
+      }
+    },
+    "mean_tokens_full": 710.6,
+    "pct_truncated": 21.93
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9619065145380935,
+    "precision": 0.8794342684906098,
+    "recall": 0.9740626605033385,
+    "f1": 0.9243328865602534,
+    "specificity": 0.9580915538362347,
+    "false_positive_rate": 0.041908446163765316,
+    "false_negative_rate": 0.02593733949666153,
+    "confusion_matrix": {
+      "tn": 11888,
+      "fp": 520,
+      "fn": 101,
+      "tp": 3793
+    },
+    "n": 16302,
+    "n_positive": 3894,
+    "n_negative": 12408
+  },
+  "threshold_free": {
+    "roc_auc": 0.9946343971962354,
+    "pr_auc": 0.9847899987683836,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.8954802259887006,
+      "fpr": 0.009993552546744036,
+      "threshold": 0.8660591840744019,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.6497175141242938,
+      "fpr": 0.0009671179883945841,
+      "threshold": 0.9728788733482361,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.6815109252929688,
+      "f1": 0.9397284608552213,
+      "precision": 0.9287684976172561,
+      "recall": 0.9509501797637391
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 3793,
+        "predicted_phishing": 974,
+        "predicted_spam": 2819
+      },
+      "false_positives_flagged": {
+        "n": 520,
+        "predicted_phishing": 175,
+        "predicted_spam": 345
+      },
+      "missed_positives": {
+        "n": 101,
+        "predicted_phishing": 35,
+        "predicted_spam": 66
+      },
+      "all_flagged": {
+        "n": 4313,
+        "predicted_phishing": 1149,
+        "predicted_spam": 3164
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 3793,
+        "predicted_phishing": 974,
+        "predicted_spam": 2819
+      }
+    },
+    "rows_with_full_label": 12408,
+    "rows_unwanted_only": 3894,
+    "evaluated": false,
+    "reason": "no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)"
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 129,
+      "bank account": 78,
+      "security alert": 70,
+      "within 24 hours": 65,
+      "urgently": 51,
+      "paypal account": 20,
+      "within 48 hours": 7,
+      "verify your account": 7,
+      "update your account": 6,
+      "account will be suspended": 5,
+      "ebay account": 5,
+      "enter your password": 5,
+      "wire transfer": 5,
+      "unusual activity": 5,
+      "confirm your identity": 4,
+      "verify your identity": 4,
+      "click here to verify": 3,
+      "log in to your account": 2,
+      "confirm your account": 2,
+      "security notice": 1,
+      "gift card": 1,
+      "reset your password": 1,
+      "account has been locked": 1,
+      "update your information": 1,
+      "account has been limited": 1,
+      "login to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 5,
+        "negative": 0
+      },
+      "update your account": {
+        "positive": 6,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 0,
+        "negative": 1
+      },
+      "click here to verify": {
+        "positive": 3,
+        "negative": 0
+      },
+      "account has been limited": {
+        "positive": 1,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 4,
+        "negative": 1
+      },
+      "account has been locked": {
+        "positive": 1,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 2,
+        "negative": 3
+      },
+      "paypal account": {
+        "positive": 20,
+        "negative": 0
+      },
+      "ebay account": {
+        "positive": 5,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "wire transfer": {
+        "positive": 5,
+        "negative": 0
+      },
+      "gift card": {
+        "positive": 0,
+        "negative": 1
+      },
+      "urgent": {
+        "positive": 81,
+        "negative": 48
+      },
+      "urgently": {
+        "positive": 22,
+        "negative": 29
+      },
+      "security alert": {
+        "positive": 2,
+        "negative": 68
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "login to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 71,
+        "negative": 7
+      },
+      "within 24 hours": {
+        "positive": 63,
+        "negative": 2
+      },
+      "within 48 hours": {
+        "positive": 4,
+        "negative": 3
+      }
+    },
+    "label_purity": 0.6106,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 155,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 243
+    },
+    "threeway_accuracy": null,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 0.9824120603015075,
+      "precision": 0.9758064516129032,
+      "recall": 0.9958847736625515,
+      "f1": 0.9857433808553973,
+      "specificity": 0.9612903225806452,
+      "false_positive_rate": 0.03870967741935484,
+      "false_negative_rate": 0.00411522633744856,
+      "confusion_matrix": {
+        "tn": 149,
+        "fp": 6,
+        "fn": 1,
+        "tp": 242
+      },
+      "n": 398,
+      "n_positive": 243,
+      "n_negative": 155,
+      "roc_auc": 0.9984866587017125
+    }
+  },
+  "per_source": {
+    "TREC-06": {
+      "threshold": 0.5,
+      "accuracy": 0.9619065145380935,
+      "precision": 0.8794342684906098,
+      "recall": 0.9740626605033385,
+      "f1": 0.9243328865602534,
+      "specificity": 0.9580915538362347,
+      "false_positive_rate": 0.041908446163765316,
+      "false_negative_rate": 0.02593733949666153,
+      "confusion_matrix": {
+        "tn": 11888,
+        "fp": 520,
+        "fn": 101,
+        "tp": 3793
+      },
+      "n": 16302,
+      "n_positive": 3894,
+      "n_negative": 12408,
+      "roc_auc": 0.9946343971962354,
+      "label3_counts": {
+        "legit": 12408,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 3894
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 3793,
+        "mean_tokens_full": 283.9,
+        "mean_tokens_fed": 232.2,
+        "median_tokens_full": 207.0,
+        "pct_truncated": 15.3
+      },
+      "tn": {
+        "n": 11888,
+        "mean_tokens_full": 827.0,
+        "mean_tokens_fed": 294.4,
+        "median_tokens_full": 269.0,
+        "pct_truncated": 23.8
+      },
+      "fp": {
+        "n": 520,
+        "mean_tokens_full": 1237.2,
+        "mean_tokens_fed": 276.6,
+        "median_tokens_full": 241.5,
+        "pct_truncated": 29.0
+      },
+      "fn": {
+        "n": 101,
+        "mean_tokens_full": 321.7,
+        "mean_tokens_fed": 220.4,
+        "median_tokens_full": 181.0,
+        "pct_truncated": 12.9
+      },
+      "correct": {
+        "n": 15681,
+        "mean_tokens_full": 695.6,
+        "mean_tokens_fed": 279.4,
+        "median_tokens_full": 247.0,
+        "pct_truncated": 21.8
+      },
+      "wrong": {
+        "n": 621,
+        "mean_tokens_full": 1088.3,
+        "mean_tokens_fed": 267.5,
+        "median_tokens_full": 213.0,
+        "pct_truncated": 26.4
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 1363,
+        "n_positive": 587,
+        "positive_rate": 0.43066764490095377,
+        "recall": 0.9744463373083475,
+        "fpr": 0.10695876288659793,
+        "accuracy": 0.9280997798972854
+      },
+      {
+        "bucket": "65-128",
+        "n": 2538,
+        "n_positive": 736,
+        "positive_rate": 0.2899921197793538,
+        "recall": 0.9619565217391305,
+        "fpr": 0.039400665926748055,
+        "accuracy": 0.9609929078014184
+      },
+      {
+        "bucket": "129-256",
+        "n": 4594,
+        "n_positive": 1196,
+        "positive_rate": 0.260339573356552,
+        "recall": 0.9765886287625418,
+        "fpr": 0.034137728075338436,
+        "accuracy": 0.9686547670875054
+      },
+      {
+        "bucket": "257-512",
+        "n": 4232,
+        "n_positive": 780,
+        "positive_rate": 0.1843100189035917,
+        "recall": 0.9782051282051282,
+        "fpr": 0.028679026651216685,
+        "accuracy": 0.9725897920604915
+      },
+      {
+        "bucket": ">512",
+        "n": 3575,
+        "n_positive": 595,
+        "positive_rate": 0.16643356643356644,
+        "recall": 0.9781512605042016,
+        "fpr": 0.05067114093959731,
+        "accuracy": 0.9541258741258741
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 16302,
+    "seconds": 57.64,
+    "rows_per_s": 282.8,
+    "padded_tokens": 4606544,
+    "real_tokens": 4546874,
+    "padded_tokens_per_s": 79918,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 10.34,
+      "p50_ms": 9.01,
+      "p95_ms": 13.85,
+      "max_ms": 19.46,
+      "avg_s": 0.0103,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.14,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 357.98,
+      "p50_ms": 340.34,
+      "p95_ms": 552.22,
+      "max_ms": 623.88,
+      "avg_s": 0.358,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 4.838,
+      "threads": 16,
+      "estimated_full_run_s": 33.6
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+    "run_name": "ft-focal",
+    "params": 337444867,
+    "timestamp": "2026-10-09 07:22",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-focal/error_analysis.md",
+    "n_fn": 101,
+    "n_fp": 520,
+    "composition": {
+      "fp": {
+        "n": 520,
+        "mailing_list_or_digest": 0.196,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.023,
+        "truncated_gt512": 0.29,
+        "lure_phrase": 0.012
+      },
+      "tn": {
+        "n": 11888,
+        "mailing_list_or_digest": 0.297,
+        "html_heavy": 0.002,
+        "very_short_lt20_tokens": 0.004,
+        "truncated_gt512": 0.238,
+        "lure_phrase": 0.013
+      },
+      "fn": {
+        "n": 101,
+        "mailing_list_or_digest": 0.089,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.129,
+        "lure_phrase": 0.01
+      },
+      "tp": {
+        "n": 3793,
+        "mailing_list_or_digest": 0.028,
+        "html_heavy": 0.001,
+        "very_short_lt20_tokens": 0.002,
+        "truncated_gt512": 0.153,
+        "lure_phrase": 0.064
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "smailex",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 934.36
+      },
+      {
+        "token": "scanning",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 934.36
+      },
+      {
+        "token": "progra",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 934.36
+      },
+      {
+        "token": "scanmail",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 934.36
+      },
+      {
+        "token": "blocki",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 934.36
+      },
+      {
+        "token": "matched",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 311.45
+      },
+      {
+        "token": "tex-fonts",
+        "share_a": 0.079,
+        "share_b": 0.0,
+        "ratio": 211.79
+      },
+      {
+        "token": "blocking",
+        "share_a": 0.129,
+        "share_b": 0.001,
+        "ratio": 201.82
+      },
+      {
+        "token": "pattern",
+        "share_a": 0.129,
+        "share_b": 0.001,
+        "ratio": 201.82
+      },
+      {
+        "token": "tpn",
+        "share_a": 0.069,
+        "share_b": 0.001,
+        "ratio": 112.12
+      },
+      {
+        "token": "setting",
+        "share_a": 0.129,
+        "share_b": 0.001,
+        "ratio": 91.74
+      },
+      {
+        "token": "pif",
+        "share_a": 0.079,
+        "share_b": 0.001,
+        "ratio": 70.6
+      },
+      {
+        "token": "utah",
+        "share_a": 0.119,
+        "share_b": 0.002,
+        "ratio": 62.29
+      },
+      {
+        "token": "detected",
+        "share_a": 0.119,
+        "share_b": 0.002,
+        "ratio": 62.29
+      },
+      {
+        "token": "attachment",
+        "share_a": 0.149,
+        "share_b": 0.003,
+        "ratio": 50.37
+      },
+      {
+        "token": "math",
+        "share_a": 0.129,
+        "share_b": 0.003,
+        "ratio": 48.05
+      },
+      {
+        "token": "enroll",
+        "share_a": 0.05,
+        "share_b": 0.001,
+        "ratio": 45.68
+      },
+      {
+        "token": "seismic",
+        "share_a": 0.05,
+        "share_b": 0.001,
+        "ratio": 45.68
+      },
+      {
+        "token": "settings",
+        "share_a": 0.129,
+        "share_b": 0.003,
+        "ratio": 37.37
+      },
+      {
+        "token": "moved",
+        "share_a": 0.119,
+        "share_b": 0.003,
+        "ratio": 37.37
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "irc-list-web",
+        "share_a": 0.054,
+        "share_b": 0.0,
+        "ratio": 1301.91
+      },
+      {
+        "token": "quarantine",
+        "share_a": 0.054,
+        "share_b": 0.0,
+        "ratio": 118.36
+      },
+      {
+        "token": "foundstone",
+        "share_a": 0.05,
+        "share_b": 0.001,
+        "ratio": 80.7
+      },
+      {
+        "token": "skidmore",
+        "share_a": 0.054,
+        "share_b": 0.001,
+        "ratio": 76.58
+      },
+      {
+        "token": "monty",
+        "share_a": 0.052,
+        "share_b": 0.001,
+        "ratio": 66.12
+      },
+      {
+        "token": "bn3",
+        "share_a": 0.06,
+        "share_b": 0.002,
+        "ratio": 27.15
+      },
+      {
+        "token": "ips",
+        "share_a": 0.054,
+        "share_b": 0.002,
+        "ratio": 26.57
+      },
+      {
+        "token": "anyplace",
+        "share_a": 0.06,
+        "share_b": 0.002,
+        "ratio": 26.16
+      },
+      {
+        "token": "noc's",
+        "share_a": 0.054,
+        "share_b": 0.002,
+        "ratio": 23.67
+      },
+      {
+        "token": "https",
+        "share_a": 0.108,
+        "share_b": 0.006,
+        "ratio": 19.12
+      },
+      {
+        "token": "cgi",
+        "share_a": 0.06,
+        "share_b": 0.004,
+        "ratio": 16.17
+      },
+      {
+        "token": "tcpview",
+        "share_a": 0.054,
+        "share_b": 0.003,
+        "ratio": 15.69
+      },
+      {
+        "token": "procexp",
+        "share_a": 0.054,
+        "share_b": 0.003,
+        "ratio": 15.69
+      },
+      {
+        "token": "sysinternals",
+        "share_a": 0.054,
+        "share_b": 0.004,
+        "ratio": 15.32
+      },
+      {
+        "token": "filtered",
+        "share_a": 0.052,
+        "share_b": 0.003,
+        "ratio": 15.14
+      },
+      {
+        "token": "atkins",
+        "share_a": 0.054,
+        "share_b": 0.004,
+        "ratio": 14.63
+      },
+      {
+        "token": "rogue",
+        "share_a": 0.056,
+        "share_b": 0.004,
+        "ratio": 14.49
+      },
+      {
+        "token": "firewall",
+        "share_a": 0.071,
+        "share_b": 0.005,
+        "ratio": 13.93
+      },
+      {
+        "token": "inbox",
+        "share_a": 0.058,
+        "share_b": 0.004,
+        "ratio": 13.79
+      },
+      {
+        "token": "manage",
+        "share_a": 0.079,
+        "share_b": 0.006,
+        "ratio": 13.64
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:052869",
+      "primary:080991",
+      "primary:086987",
+      "primary:056767",
+      "primary:121268",
+      "primary:173636",
+      "primary:020947",
+      "primary:072993",
+      "primary:044992",
+      "primary:024565",
+      "primary:076023",
+      "primary:129745",
+      "primary:071577",
+      "primary:173938",
+      "primary:030013",
+      "primary:063077",
+      "primary:145723",
+      "primary:154489",
+      "primary:186523",
+      "primary:147887",
+      "primary:033014",
+      "primary:156328",
+      "primary:143065",
+      "primary:147193",
+      "primary:101618"
+    ],
+    "top_fp_doc_ids": [
+      "primary:015304",
+      "primary:040048",
+      "primary:194751",
+      "primary:072489",
+      "primary:194063",
+      "primary:033174",
+      "primary:101560",
+      "primary:159590",
+      "primary:184838",
+      "primary:011014",
+      "primary:193401",
+      "primary:072275",
+      "primary:170605",
+      "primary:175533",
+      "primary:131182",
+      "primary:106445",
+      "primary:078484",
+      "primary:196901",
+      "primary:143547",
+      "primary:129925",
+      "primary:092561",
+      "primary:182736",
+      "primary:158244",
+      "primary:152569",
+      "primary:011118"
+    ],
+    "threeway": null
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-focal/confusion_matrix.png",
+    "roc_curve": "reports/eval_ft-focal/roc_curve.png",
+    "pr_curve": "reports/eval_ft-focal/pr_curve.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-focal/metrics_test3.json b/presentation/reports/eval_ft-focal/metrics_test3.json
new file mode 100644
index 0000000000000000000000000000000000000000..4741d3afc63a0a6f1b5a258363ec3984594a7a92
--- /dev/null
+++ b/presentation/reports/eval_ft-focal/metrics_test3.json
@@ -0,0 +1,1667 @@
+{
+  "timestamp": "2026-10-09 07:22",
+  "run_name": "ft-focal",
+  "split": "test3",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout3.parquet",
+  "split_title": "three-way holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-focal",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "focal",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 1,
+    "best_step": 8750,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9951084002657166,
+    "val_metrics": {
+      "accuracy": 0.9947003402250719,
+      "precision": 0.9954089646007007,
+      "recall": 0.9948080173871046,
+      "f1": 0.9951084002657166,
+      "roc_auc": 0.9998186946414248,
+      "pr_auc": 0.9998464119156345,
+      "tn": 6964,
+      "fp": 38,
+      "fn": 43,
+      "tp": 8239,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.42074909806251526,
+      "f1_at_optimal_threshold": 0.9952995058454863,
+      "f1_unwanted": 0.9951084002657166,
+      "n_3way": 8105,
+      "accuracy_3way": 0.994694632942628,
+      "macro_f1_3way": 0.9868751212108838,
+      "precision_legitimate": 0.9988533753762362,
+      "recall_legitimate": 0.99528706083976,
+      "f1_legitimate": 0.9970670291151013,
+      "precision_phishing": 0.9853372434017595,
+      "recall_phishing": 0.9882352941176471,
+      "f1_phishing": 0.986784140969163,
+      "precision_spam": 0.9618805590851334,
+      "recall_spam": 0.9921363040629095,
+      "f1_spam": 0.9767741935483871,
+      "cm3_ll": 6969,
+      "cm3_lp": 4,
+      "cm3_ls": 29,
+      "cm3_pl": 3,
+      "cm3_pp": 336,
+      "cm3_ps": 1,
+      "cm3_sl": 5,
+      "cm3_sp": 1,
+      "cm3_ss": 757,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 339,
+      "pvs_phish_as_spam": 1,
+      "pvs_spam_as_phish": 1,
+      "pvs_spam_as_spam": 762,
+      "val_loss": 0.030296484990695233,
+      "n": 15284,
+      "eval_seconds": 48.0
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9947003402250719,
+        "precision": 0.9954089646007007,
+        "recall": 0.9948080173871046,
+        "f1": 0.9951084002657166,
+        "roc_auc": 0.9998186946414248,
+        "pr_auc": 0.9998464119156345,
+        "tn": 6964,
+        "fp": 38,
+        "fn": 43,
+        "tp": 8239,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.42074909806251526,
+        "f1_at_optimal_threshold": 0.9952995058454863,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.994694632942628,
+        "macro_f1": 0.9868751212108838,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9988533753762362,
+            "recall": 0.99528706083976,
+            "f1": 0.9970670291151013
+          },
+          "Phishing": {
+            "precision": 0.9853372434017595,
+            "recall": 0.9882352941176471,
+            "f1": 0.986784140969163
+          },
+          "Spam": {
+            "precision": 0.9618805590851334,
+            "recall": 0.9921363040629095,
+            "f1": 0.9767741935483871
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6969,
+              4,
+              29
+            ],
+            [
+              3,
+              336,
+              1
+            ],
+            [
+              5,
+              1,
+              757
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              339,
+              1
+            ],
+            [
+              1,
+              762
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.42074909806251526,
+      "val_f1_at_optimal": 0.9952995058454863
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 3220,
+    "positive": 1221,
+    "negative": 1999,
+    "positive_rate": 0.3792,
+    "label3": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "by_source": {
+      "Assassin": {
+        "rows": 236,
+        "positive": 72,
+        "label3": {
+          "legit": 164,
+          "phish": 0,
+          "spam": 72,
+          "unwanted": 0
+        }
+      },
+      "CEAS-08": {
+        "rows": 736,
+        "positive": 0,
+        "label3": {
+          "legit": 736,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Enron": {
+        "rows": 628,
+        "positive": 628,
+        "label3": {
+          "legit": 0,
+          "phish": 0,
+          "spam": 628,
+          "unwanted": 0
+        }
+      },
+      "Nazario": {
+        "rows": 222,
+        "positive": 222,
+        "label3": {
+          "legit": 0,
+          "phish": 222,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Nigerian_Fraud": {
+        "rows": 299,
+        "positive": 299,
+        "label3": {
+          "legit": 0,
+          "phish": 299,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "TREC-07": {
+        "rows": 1099,
+        "positive": 0,
+        "label3": {
+          "legit": 1099,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      }
+    },
+    "mean_tokens_full": 835.2,
+    "pct_truncated": 34.01
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.989751552795031,
+    "precision": 0.9892915980230642,
+    "recall": 0.9836199836199836,
+    "f1": 0.986447638603696,
+    "specificity": 0.9934967483741871,
+    "false_positive_rate": 0.0065032516258129065,
+    "false_negative_rate": 0.01638001638001638,
+    "confusion_matrix": {
+      "tn": 1986,
+      "fp": 13,
+      "fn": 20,
+      "tp": 1201
+    },
+    "n": 3220,
+    "n_positive": 1221,
+    "n_negative": 1999
+  },
+  "threshold_free": {
+    "roc_auc": 0.9993325901279878,
+    "pr_auc": 0.9988537821967886,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.9868959868959869,
+      "fpr": 0.009004502251125562,
+      "threshold": 0.43377745151519775,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.8787878787878788,
+      "fpr": 0.0005002501250625312,
+      "threshold": 0.9472352862358093,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.47499534487724304,
+      "f1": 0.9876948318293682,
+      "precision": 0.9893179950698439,
+      "recall": 0.9860769860769861
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 1201,
+        "predicted_phishing": 515,
+        "predicted_spam": 686
+      },
+      "false_positives_flagged": {
+        "n": 13,
+        "predicted_phishing": 1,
+        "predicted_spam": 12
+      },
+      "missed_positives": {
+        "n": 20,
+        "predicted_phishing": 3,
+        "predicted_spam": 17
+      },
+      "all_flagged": {
+        "n": 1214,
+        "predicted_phishing": 516,
+        "predicted_spam": 698
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 0,
+        "predicted_phishing": 0,
+        "predicted_spam": 0
+      }
+    },
+    "rows_with_full_label": 3220,
+    "rows_unwanted_only": 0,
+    "evaluated": true,
+    "reason": null,
+    "confusion_matrix": {
+      "labels": [
+        "Legitimate",
+        "Phishing",
+        "Spam"
+      ],
+      "matrix": [
+        [
+          1986,
+          1,
+          12
+        ],
+        [
+          4,
+          514,
+          3
+        ],
+        [
+          16,
+          1,
+          683
+        ]
+      ],
+      "cells": {
+        "true_legitimate": {
+          "pred_legitimate": 1986,
+          "pred_phishing": 1,
+          "pred_spam": 12
+        },
+        "true_phishing": {
+          "pred_legitimate": 4,
+          "pred_phishing": 514,
+          "pred_spam": 3
+        },
+        "true_spam": {
+          "pred_legitimate": 16,
+          "pred_phishing": 1,
+          "pred_spam": 683
+        }
+      }
+    },
+    "accuracy": 0.9885093167701864,
+    "macro_f1": 0.9867305252015051,
+    "macro_f1_present_classes": 0.9867305252015051,
+    "classes_present": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "per_class": {
+      "Legitimate": {
+        "precision": 0.9900299102691924,
+        "recall": 0.9934967483741871,
+        "f1": 0.9917602996254682,
+        "support": 1999
+      },
+      "Phishing": {
+        "precision": 0.9961240310077519,
+        "recall": 0.9865642994241842,
+        "f1": 0.991321118611379,
+        "support": 521
+      },
+      "Spam": {
+        "precision": 0.9785100286532952,
+        "recall": 0.9757142857142858,
+        "f1": 0.977110157367668,
+        "support": 700
+      }
+    },
+    "argmax_rule": {
+      "doc": "plain argmax over the three probabilities (secondary; can disagree with the binary verdict near the threshold)",
+      "accuracy": 0.987888198757764,
+      "macro_f1": 0.9858185605052249,
+      "per_class": {
+        "Legitimate": {
+          "precision": 0.9880715705765407,
+          "recall": 0.9944972486243121,
+          "f1": 0.9912739965095986,
+          "support": 1999
+        },
+        "Phishing": {
+          "precision": 0.9980430528375733,
+          "recall": 0.9788867562380038,
+          "f1": 0.9883720930232558,
+          "support": 521
+        },
+        "Spam": {
+          "precision": 0.9799139167862266,
+          "recall": 0.9757142857142858,
+          "f1": 0.9778095919828204,
+          "support": 700
+        }
+      },
+      "rows_where_rules_differ": 6
+    },
+    "phish_vs_spam_on_true_unwanted": {
+      "n_true_unwanted": 1221,
+      "kind_only": {
+        "doc": "given the row is unwanted, is argmax(P(phishing), P(spam)) right? (independent of the roll-up threshold)",
+        "labels": [
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            517,
+            4
+          ],
+          [
+            1,
+            699
+          ]
+        ],
+        "cells": {
+          "true_phishing": {
+            "pred_phishing": 517,
+            "pred_spam": 4
+          },
+          "true_spam": {
+            "pred_phishing": 1,
+            "pred_spam": 699
+          }
+        },
+        "accuracy": 0.9959049959049959,
+        "phishing_as_spam": 4,
+        "spam_as_phishing": 1
+      },
+      "rollup_rule": {
+        "doc": "rows = true Phishing / Spam; columns = predicted Legitimate (missed) / Phishing / Spam under the roll-up rule",
+        "rows": [
+          "Phishing",
+          "Spam"
+        ],
+        "columns": [
+          "Legitimate",
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            4,
+            514,
+            3
+          ],
+          [
+            16,
+            1,
+            683
+          ]
+        ],
+        "missed_as_legitimate": {
+          "phishing": 4,
+          "spam": 16
+        }
+      }
+    },
+    "per_source": {
+      "Assassin": {
+        "n": 236,
+        "counts": {
+          "Legitimate": 164,
+          "Phishing": 0,
+          "Spam": 72
+        },
+        "accuracy": 0.9533898305084746,
+        "macro_f1": 0.6305609284332688,
+        "macro_f1_present_classes": 0.9458413926499033,
+        "per_class_recall": {
+          "Legitimate": 0.975609756097561,
+          "Spam": 0.9027777777777778
+        }
+      },
+      "CEAS-08": {
+        "n": 736,
+        "counts": {
+          "Legitimate": 736,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9904891304347826,
+        "macro_f1": 0.331740614334471,
+        "macro_f1_present_classes": 0.995221843003413,
+        "per_class_recall": {
+          "Legitimate": 0.9904891304347826
+        }
+      },
+      "Enron": {
+        "n": 628,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 0,
+          "Spam": 628
+        },
+        "accuracy": 0.9840764331210191,
+        "macro_f1": 0.3306581059390048,
+        "macro_f1_present_classes": 0.9919743178170144,
+        "per_class_recall": {
+          "Spam": 0.9840764331210191
+        }
+      },
+      "Nazario": {
+        "n": 222,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 222,
+          "Spam": 0
+        },
+        "accuracy": 0.9864864864864865,
+        "macro_f1": 0.3310657596371882,
+        "macro_f1_present_classes": 0.9931972789115646,
+        "per_class_recall": {
+          "Phishing": 0.9864864864864865
+        }
+      },
+      "Nigerian_Fraud": {
+        "n": 299,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 299,
+          "Spam": 0
+        },
+        "accuracy": 0.9866220735785953,
+        "macro_f1": 0.33108866442199775,
+        "macro_f1_present_classes": 0.9932659932659933,
+        "per_class_recall": {
+          "Phishing": 0.9866220735785953
+        }
+      },
+      "TREC-07": {
+        "n": 1099,
+        "counts": {
+          "Legitimate": 1099,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9981801637852593,
+        "macro_f1": 0.33302975106253796,
+        "macro_f1_present_classes": 0.9990892531876139,
+        "per_class_recall": {
+          "Legitimate": 0.9981801637852593
+        }
+      }
+    }
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 139,
+      "bank account": 91,
+      "urgently": 25,
+      "verify your account": 15,
+      "confirm your account": 9,
+      "update your account": 9,
+      "within 24 hours": 8,
+      "security alert": 8,
+      "paypal account": 7,
+      "password will expire": 7,
+      "click here to verify": 6,
+      "unusual activity": 4,
+      "wire transfer": 3,
+      "update your information": 3,
+      "gift card": 3,
+      "verify your identity": 2,
+      "unusual sign-in": 2,
+      "account will be suspended": 2,
+      "update your payment": 2,
+      "failure to verify": 2,
+      "within 48 hours": 2,
+      "confirm your identity": 2,
+      "enter your password": 1,
+      "update your billing": 1,
+      "reset your password": 1,
+      "suspicious activity": 1,
+      "security notice": 1,
+      "log in to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 15,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "update your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 2,
+        "negative": 1
+      },
+      "update your billing": {
+        "positive": 1,
+        "negative": 0
+      },
+      "update your payment": {
+        "positive": 2,
+        "negative": 0
+      },
+      "click here to verify": {
+        "positive": 6,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual sign-in": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "suspicious activity": {
+        "positive": 1,
+        "negative": 0
+      },
+      "paypal account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "password will expire": {
+        "positive": 7,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 1,
+        "negative": 0
+      },
+      "wire transfer": {
+        "positive": 2,
+        "negative": 1
+      },
+      "gift card": {
+        "positive": 3,
+        "negative": 0
+      },
+      "urgent": {
+        "positive": 133,
+        "negative": 6
+      },
+      "urgently": {
+        "positive": 25,
+        "negative": 0
+      },
+      "security alert": {
+        "positive": 8,
+        "negative": 0
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 91,
+        "negative": 0
+      },
+      "failure to verify": {
+        "positive": 2,
+        "negative": 0
+      },
+      "within 24 hours": {
+        "positive": 8,
+        "negative": 0
+      },
+      "within 48 hours": {
+        "positive": 2,
+        "negative": 0
+      }
+    },
+    "label_purity": 0.9745,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 7,
+      "phish": 236,
+      "spam": 32,
+      "unwanted": 0
+    },
+    "threeway_accuracy": 0.9927272727272727,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 1.0,
+      "precision": 1.0,
+      "recall": 1.0,
+      "f1": 1.0,
+      "specificity": 1.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 7,
+        "fp": 0,
+        "fn": 0,
+        "tp": 268
+      },
+      "n": 275,
+      "n_positive": 268,
+      "n_negative": 7,
+      "roc_auc": 1.0
+    }
+  },
+  "per_source": {
+    "Assassin": {
+      "threshold": 0.5,
+      "accuracy": 0.9576271186440678,
+      "precision": 0.9428571428571428,
+      "recall": 0.9166666666666666,
+      "f1": 0.9295774647887323,
+      "specificity": 0.975609756097561,
+      "false_positive_rate": 0.024390243902439025,
+      "false_negative_rate": 0.08333333333333333,
+      "confusion_matrix": {
+        "tn": 160,
+        "fp": 4,
+        "fn": 6,
+        "tp": 66
+      },
+      "n": 236,
+      "n_positive": 72,
+      "n_negative": 164,
+      "roc_auc": 0.994579945799458,
+      "label3_counts": {
+        "legit": 164,
+        "phish": 0,
+        "spam": 72,
+        "unwanted": 0
+      }
+    },
+    "CEAS-08": {
+      "threshold": 0.5,
+      "accuracy": 0.9904891304347826,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9904891304347826,
+      "false_positive_rate": 0.009510869565217392,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 729,
+        "fp": 7,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 736,
+      "n_positive": 0,
+      "n_negative": 736,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 736,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Enron": {
+      "threshold": 0.5,
+      "accuracy": 0.9840764331210191,
+      "precision": 1.0,
+      "recall": 0.9840764331210191,
+      "f1": 0.9919743178170145,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.01592356687898089,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 10,
+        "tp": 618
+      },
+      "n": 628,
+      "n_positive": 628,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 0,
+        "spam": 628,
+        "unwanted": 0
+      }
+    },
+    "Nazario": {
+      "threshold": 0.5,
+      "accuracy": 0.9864864864864865,
+      "precision": 1.0,
+      "recall": 0.9864864864864865,
+      "f1": 0.9931972789115647,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.013513513513513514,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 3,
+        "tp": 219
+      },
+      "n": 222,
+      "n_positive": 222,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 222,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Nigerian_Fraud": {
+      "threshold": 0.5,
+      "accuracy": 0.9966555183946488,
+      "precision": 1.0,
+      "recall": 0.9966555183946488,
+      "f1": 0.998324958123953,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.0033444816053511705,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 1,
+        "tp": 298
+      },
+      "n": 299,
+      "n_positive": 299,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 299,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "TREC-07": {
+      "threshold": 0.5,
+      "accuracy": 0.9981801637852593,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9981801637852593,
+      "false_positive_rate": 0.0018198362147406734,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 1097,
+        "fp": 2,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 1099,
+      "n_positive": 0,
+      "n_negative": 1099,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 1099,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 1201,
+        "mean_tokens_full": 978.0,
+        "mean_tokens_fed": 279.8,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.6
+      },
+      "tn": {
+        "n": 1986,
+        "mean_tokens_full": 732.6,
+        "mean_tokens_fed": 358.4,
+        "median_tokens_full": 392.5,
+        "pct_truncated": 38.3
+      },
+      "fp": {
+        "n": 13,
+        "mean_tokens_full": 1814.3,
+        "mean_tokens_fed": 377.2,
+        "median_tokens_full": 875.0,
+        "pct_truncated": 61.5
+      },
+      "fn": {
+        "n": 20,
+        "mean_tokens_full": 1807.8,
+        "mean_tokens_fed": 284.1,
+        "median_tokens_full": 291.0,
+        "pct_truncated": 35.0
+      },
+      "correct": {
+        "n": 3187,
+        "mean_tokens_full": 825.1,
+        "mean_tokens_fed": 328.8,
+        "median_tokens_full": 339.0,
+        "pct_truncated": 33.9
+      },
+      "wrong": {
+        "n": 33,
+        "mean_tokens_full": 1810.4,
+        "mean_tokens_fed": 320.8,
+        "median_tokens_full": 368.0,
+        "pct_truncated": 45.5
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "unwanted": {
+        "n": 1221,
+        "mean_tokens_full": 991.6,
+        "mean_tokens_fed": 279.9,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.7
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "phish": {
+        "n": 521,
+        "mean_tokens_full": 1349.4,
+        "mean_tokens_fed": 355.1,
+        "median_tokens_full": 412.0,
+        "pct_truncated": 39.9
+      },
+      "spam": {
+        "n": 700,
+        "mean_tokens_full": 725.3,
+        "mean_tokens_fed": 223.9,
+        "median_tokens_full": 159.0,
+        "pct_truncated": 16.9
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 129,
+        "n_positive": 99,
+        "positive_rate": 0.7674418604651163,
+        "recall": 0.9494949494949495,
+        "fpr": 0.03333333333333333,
+        "accuracy": 0.9534883720930233
+      },
+      {
+        "bucket": "65-128",
+        "n": 427,
+        "n_positive": 252,
+        "positive_rate": 0.5901639344262295,
+        "recall": 0.9920634920634921,
+        "fpr": 0.011428571428571429,
+        "accuracy": 0.990632318501171
+      },
+      {
+        "bucket": "129-256",
+        "n": 699,
+        "n_positive": 276,
+        "positive_rate": 0.3948497854077253,
+        "recall": 0.9927536231884058,
+        "fpr": 0.0,
+        "accuracy": 0.9971387696709585
+      },
+      {
+        "bucket": "257-512",
+        "n": 870,
+        "n_positive": 268,
+        "positive_rate": 0.3080459770114943,
+        "recall": 0.9850746268656716,
+        "fpr": 0.0033222591362126247,
+        "accuracy": 0.993103448275862
+      },
+      {
+        "bucket": ">512",
+        "n": 1095,
+        "n_positive": 326,
+        "positive_rate": 0.29771689497716897,
+        "recall": 0.9785276073619632,
+        "fpr": 0.010403120936280884,
+        "accuracy": 0.9863013698630136
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 3220,
+    "seconds": 13.49,
+    "rows_per_s": 238.7,
+    "padded_tokens": 1079936,
+    "real_tokens": 1058490,
+    "padded_tokens_per_s": 80062,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 10.34,
+      "p50_ms": 9.01,
+      "p95_ms": 13.85,
+      "max_ms": 19.46,
+      "avg_s": 0.0103,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.14,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 357.98,
+      "p50_ms": 340.34,
+      "p95_ms": 552.22,
+      "max_ms": 623.88,
+      "avg_s": 0.358,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 4.838,
+      "threads": 16,
+      "estimated_full_run_s": 33.6
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+    "run_name": "ft-focal",
+    "params": 337444867,
+    "timestamp": "2026-10-09 07:22",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-focal/error_analysis_test3.md",
+    "n_fn": 20,
+    "n_fp": 13,
+    "composition": {
+      "fp": {
+        "n": 13,
+        "mailing_list_or_digest": 0.462,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.615,
+        "lure_phrase": 0.0
+      },
+      "tn": {
+        "n": 1986,
+        "mailing_list_or_digest": 0.563,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.383,
+        "lure_phrase": 0.004
+      },
+      "fn": {
+        "n": 20,
+        "mailing_list_or_digest": 0.3,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.1,
+        "truncated_gt512": 0.35,
+        "lure_phrase": 0.0
+      },
+      "tp": {
+        "n": 1201,
+        "mailing_list_or_digest": 0.083,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.006,
+        "truncated_gt512": 0.266,
+        "lure_phrase": 0.223
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "invite",
+        "share_a": 0.25,
+        "share_b": 0.002,
+        "ratio": 128.94
+      },
+      {
+        "token": "practical",
+        "share_a": 0.15,
+        "share_b": 0.002,
+        "ratio": 82.05
+      },
+      {
+        "token": "cut",
+        "share_a": 0.15,
+        "share_b": 0.002,
+        "ratio": 82.05
+      },
+      {
+        "token": "networking",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 75.36
+      },
+      {
+        "token": "audience",
+        "share_a": 0.15,
+        "share_b": 0.003,
+        "ratio": 45.59
+      },
+      {
+        "token": "oct",
+        "share_a": 0.15,
+        "share_b": 0.003,
+        "ratio": 45.59
+      },
+      {
+        "token": "innovative",
+        "share_a": 0.15,
+        "share_b": 0.003,
+        "ratio": 45.59
+      },
+      {
+        "token": "event",
+        "share_a": 0.2,
+        "share_b": 0.005,
+        "ratio": 40.58
+      },
+      {
+        "token": "angeles",
+        "share_a": 0.15,
+        "share_b": 0.004,
+        "ratio": 37.3
+      },
+      {
+        "token": "anyway",
+        "share_a": 0.15,
+        "share_b": 0.004,
+        "ratio": 37.3
+      },
+      {
+        "token": "feature",
+        "share_a": 0.15,
+        "share_b": 0.005,
+        "ratio": 31.56
+      },
+      {
+        "token": "york",
+        "share_a": 0.15,
+        "share_b": 0.005,
+        "ratio": 31.56
+      },
+      {
+        "token": "subscription",
+        "share_a": 0.2,
+        "share_b": 0.007,
+        "ratio": 27.76
+      },
+      {
+        "token": "technologies",
+        "share_a": 0.15,
+        "share_b": 0.006,
+        "ratio": 27.35
+      },
+      {
+        "token": "entertainment",
+        "share_a": 0.15,
+        "share_b": 0.007,
+        "ratio": 24.13
+      },
+      {
+        "token": "staff",
+        "share_a": 0.2,
+        "share_b": 0.011,
+        "ratio": 19.54
+      },
+      {
+        "token": "association",
+        "share_a": 0.15,
+        "share_b": 0.008,
+        "ratio": 19.54
+      },
+      {
+        "token": "hosting",
+        "share_a": 0.15,
+        "share_b": 0.009,
+        "ratio": 17.84
+      },
+      {
+        "token": "markets",
+        "share_a": 0.15,
+        "share_b": 0.009,
+        "ratio": 17.84
+      },
+      {
+        "token": "mailman",
+        "share_a": 0.15,
+        "share_b": 0.009,
+        "ratio": 17.84
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "outlets",
+        "share_a": 0.231,
+        "share_b": 0.001,
+        "ratio": 206.01
+      },
+      {
+        "token": "czzz-unspun",
+        "share_a": 0.231,
+        "share_b": 0.001,
+        "ratio": 206.01
+      },
+      {
+        "token": "zzz-unspun",
+        "share_a": 0.231,
+        "share_b": 0.001,
+        "ratio": 206.01
+      },
+      {
+        "token": "cook",
+        "share_a": 0.231,
+        "share_b": 0.001,
+        "ratio": 206.01
+      },
+      {
+        "token": "full-color",
+        "share_a": 0.385,
+        "share_b": 0.002,
+        "ratio": 179.85
+      },
+      {
+        "token": "irreverently",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "patel",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "jimmy",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "jimduffy86",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "vineyard",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "imakenews",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "unspun",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "guterman",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "authoritatively",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "duffy",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "muehlbauer",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "mediaunspun",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "willya",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "lori",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "comtm",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:191966",
+      "primary:148887",
+      "aura:Nazario:000097",
+      "primary:010013",
+      "primary:007176",
+      "aura:Nazario:000643",
+      "primary:181246",
+      "primary:095405",
+      "primary:014300",
+      "primary:001939",
+      "primary:033220",
+      "primary:025244",
+      "primary:133074",
+      "aura:Nigerian_Fraud:003049",
+      "aura:Nazario:000299",
+      "primary:046569",
+      "primary:135631",
+      "primary:148857",
+      "primary:129677",
+      "primary:075760"
+    ],
+    "top_fp_doc_ids": [
+      "primary:144048",
+      "primary:199386",
+      "primary:091801",
+      "primary:152763",
+      "primary:165228",
+      "primary:041713",
+      "primary:103927",
+      "primary:055965",
+      "primary:170237",
+      "primary:117398",
+      "primary:168120",
+      "primary:057514",
+      "primary:021911"
+    ],
+    "threeway": {
+      "n_phish_as_spam": 3,
+      "n_spam_as_phish": 1,
+      "top_phish_as_spam_doc_ids": [
+        "aura:Nigerian_Fraud:001698",
+        "aura:Nigerian_Fraud:000337",
+        "aura:Nigerian_Fraud:000824"
+      ],
+      "top_spam_as_phish_doc_ids": [
+        "primary:077835"
+      ],
+      "phish_as_spam_tokens": [
+        {
+          "token": "join",
+          "share_a": 1.0,
+          "share_b": 0.019,
+          "ratio": 49.0
+        },
+        {
+          "token": "could",
+          "share_a": 1.0,
+          "share_b": 0.076,
+          "ratio": 13.03
+        },
+        {
+          "token": "www",
+          "share_a": 1.0,
+          "share_b": 0.15,
+          "ratio": 6.64
+        },
+        {
+          "token": "new",
+          "share_a": 1.0,
+          "share_b": 0.16,
+          "ratio": 6.24
+        },
+        {
+          "token": "http",
+          "share_a": 1.0,
+          "share_b": 0.193,
+          "ratio": 5.17
+        },
+        {
+          "token": "years",
+          "share_a": 1.0,
+          "share_b": 0.196,
+          "ratio": 5.07
+        },
+        {
+          "token": "get",
+          "share_a": 1.0,
+          "share_b": 0.202,
+          "ratio": 4.92
+        },
+        {
+          "token": "about",
+          "share_a": 1.0,
+          "share_b": 0.239,
+          "ratio": 4.17
+        },
+        {
+          "token": "after",
+          "share_a": 1.0,
+          "share_b": 0.278,
+          "ratio": 3.59
+        },
+        {
+          "token": "who",
+          "share_a": 1.0,
+          "share_b": 0.317,
+          "ratio": 3.15
+        },
+        {
+          "token": "into",
+          "share_a": 1.0,
+          "share_b": 0.323,
+          "ratio": 3.09
+        },
+        {
+          "token": "com",
+          "share_a": 1.0,
+          "share_b": 0.339,
+          "ratio": 2.95
+        },
+        {
+          "token": "which",
+          "share_a": 1.0,
+          "share_b": 0.362,
+          "ratio": 2.76
+        },
+        {
+          "token": "now",
+          "share_a": 1.0,
+          "share_b": 0.379,
+          "ratio": 2.63
+        },
+        {
+          "token": "can",
+          "share_a": 1.0,
+          "share_b": 0.467,
+          "ratio": 2.14
+        },
+        {
+          "token": "are",
+          "share_a": 1.0,
+          "share_b": 0.562,
+          "ratio": 1.78
+        },
+        {
+          "token": "from",
+          "share_a": 1.0,
+          "share_b": 0.644,
+          "ratio": 1.55
+        },
+        {
+          "token": "will",
+          "share_a": 1.0,
+          "share_b": 0.714,
+          "ratio": 1.4
+        },
+        {
+          "token": "for",
+          "share_a": 1.0,
+          "share_b": 0.8,
+          "ratio": 1.25
+        },
+        {
+          "token": "the",
+          "share_a": 1.0,
+          "share_b": 0.811,
+          "ratio": 1.23
+        }
+      ],
+      "spam_as_phish_tokens": []
+    }
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-focal/confusion_matrix_test3.png",
+    "confusion_matrix3": "reports/eval_ft-focal/confusion_matrix3_test3.png",
+    "roc_curve": "reports/eval_ft-focal/roc_curve_test3.png",
+    "pr_curve": "reports/eval_ft-focal/pr_curve_test3.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-scratch/metrics.json b/presentation/reports/eval_ft-scratch/metrics.json
new file mode 100644
index 0000000000000000000000000000000000000000..01f7caf71abe9fb43cd19840724d9b09bd76e9d6
--- /dev/null
+++ b/presentation/reports/eval_ft-scratch/metrics.json
@@ -0,0 +1,1109 @@
+{
+  "timestamp": "2026-10-09 09:57",
+  "run_name": "ft-scratch",
+  "split": "test",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet",
+  "split_title": "TREC-06 binary holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-scratch/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-scratch",
+      "init": "scratch",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "cross_entropy",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 0,
+    "best_step": 5009,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9874562778916898,
+    "val_metrics": {
+      "accuracy": 0.9863909971211725,
+      "precision": 0.9863855421686747,
+      "recall": 0.9885293407389519,
+      "f1": 0.9874562778916898,
+      "roc_auc": 0.9984844430897414,
+      "pr_auc": 0.998618660839301,
+      "tn": 6889,
+      "fp": 113,
+      "fn": 95,
+      "tp": 8187,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.6013138890266418,
+      "f1_at_optimal_threshold": 0.9878557186876926,
+      "f1_unwanted": 0.9874562778916898,
+      "n_3way": 8105,
+      "accuracy_3way": 0.9832202344231955,
+      "macro_f1_3way": 0.9537815709229355,
+      "precision_legitimate": 0.9979745370370371,
+      "recall_legitimate": 0.9851471008283348,
+      "f1_legitimate": 0.991519333045853,
+      "precision_phishing": 0.9384164222873901,
+      "recall_phishing": 0.9411764705882353,
+      "f1_phishing": 0.9397944199706314,
+      "precision_spam": 0.8814553990610329,
+      "recall_spam": 0.9842726081258192,
+      "f1_spam": 0.930030959752322,
+      "cm3_ll": 6898,
+      "cm3_lp": 20,
+      "cm3_ls": 84,
+      "cm3_pl": 3,
+      "cm3_pp": 320,
+      "cm3_ps": 17,
+      "cm3_sl": 11,
+      "cm3_sp": 1,
+      "cm3_ss": 751,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 322,
+      "pvs_phish_as_spam": 18,
+      "pvs_spam_as_phish": 1,
+      "pvs_spam_as_spam": 762,
+      "val_loss": 0.04709533672780773,
+      "n": 15284,
+      "eval_seconds": 48.1
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9863909971211725,
+        "precision": 0.9863855421686747,
+        "recall": 0.9885293407389519,
+        "f1": 0.9874562778916898,
+        "roc_auc": 0.9984844430897414,
+        "pr_auc": 0.998618660839301,
+        "tn": 6889,
+        "fp": 113,
+        "fn": 95,
+        "tp": 8187,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.6013138890266418,
+        "f1_at_optimal_threshold": 0.9878557186876926,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.9832202344231955,
+        "macro_f1": 0.9537815709229355,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9979745370370371,
+            "recall": 0.9851471008283348,
+            "f1": 0.991519333045853
+          },
+          "Phishing": {
+            "precision": 0.9384164222873901,
+            "recall": 0.9411764705882353,
+            "f1": 0.9397944199706314
+          },
+          "Spam": {
+            "precision": 0.8814553990610329,
+            "recall": 0.9842726081258192,
+            "f1": 0.930030959752322
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6898,
+              20,
+              84
+            ],
+            [
+              3,
+              320,
+              17
+            ],
+            [
+              11,
+              1,
+              751
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              322,
+              18
+            ],
+            [
+              1,
+              762
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.6013138890266418,
+      "val_f1_at_optimal": 0.9878557186876926
+    },
+    "init": "scratch",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 16302,
+    "positive": 3894,
+    "negative": 12408,
+    "positive_rate": 0.2389,
+    "label3": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "by_source": {
+      "TREC-06": {
+        "rows": 16302,
+        "positive": 3894,
+        "label3": {
+          "legit": 12408,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 3894
+        }
+      }
+    },
+    "mean_tokens_full": 710.6,
+    "pct_truncated": 21.93
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9165746534167587,
+    "precision": 0.7751954821894005,
+    "recall": 0.9165382639958911,
+    "f1": 0.8399623440809603,
+    "specificity": 0.9165860735009671,
+    "false_positive_rate": 0.08341392649903288,
+    "false_negative_rate": 0.08346173600410889,
+    "confusion_matrix": {
+      "tn": 11373,
+      "fp": 1035,
+      "fn": 325,
+      "tp": 3569
+    },
+    "n": 16302,
+    "n_positive": 3894,
+    "n_negative": 12408
+  },
+  "threshold_free": {
+    "roc_auc": 0.9755163385154697,
+    "pr_auc": 0.9455998878489401,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.7773497688751926,
+      "fpr": 0.009993552546744036,
+      "threshold": 0.9809021353721619,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.4953775038520801,
+      "fpr": 0.0009671179883945841,
+      "threshold": 0.9961771965026855,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.941460371017456,
+      "f1": 0.8772071707777328,
+      "precision": 0.9231205673758865,
+      "recall": 0.8356445814072933
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 3569,
+        "predicted_phishing": 332,
+        "predicted_spam": 3237
+      },
+      "false_positives_flagged": {
+        "n": 1035,
+        "predicted_phishing": 281,
+        "predicted_spam": 754
+      },
+      "missed_positives": {
+        "n": 325,
+        "predicted_phishing": 84,
+        "predicted_spam": 241
+      },
+      "all_flagged": {
+        "n": 4604,
+        "predicted_phishing": 613,
+        "predicted_spam": 3991
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 3569,
+        "predicted_phishing": 332,
+        "predicted_spam": 3237
+      }
+    },
+    "rows_with_full_label": 12408,
+    "rows_unwanted_only": 3894,
+    "evaluated": false,
+    "reason": "no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)"
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 129,
+      "bank account": 78,
+      "security alert": 70,
+      "within 24 hours": 65,
+      "urgently": 51,
+      "paypal account": 20,
+      "within 48 hours": 7,
+      "verify your account": 7,
+      "update your account": 6,
+      "account will be suspended": 5,
+      "ebay account": 5,
+      "enter your password": 5,
+      "wire transfer": 5,
+      "unusual activity": 5,
+      "confirm your identity": 4,
+      "verify your identity": 4,
+      "click here to verify": 3,
+      "log in to your account": 2,
+      "confirm your account": 2,
+      "security notice": 1,
+      "gift card": 1,
+      "reset your password": 1,
+      "account has been locked": 1,
+      "update your information": 1,
+      "account has been limited": 1,
+      "login to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 5,
+        "negative": 0
+      },
+      "update your account": {
+        "positive": 6,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 0,
+        "negative": 1
+      },
+      "click here to verify": {
+        "positive": 3,
+        "negative": 0
+      },
+      "account has been limited": {
+        "positive": 1,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 4,
+        "negative": 1
+      },
+      "account has been locked": {
+        "positive": 1,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 2,
+        "negative": 3
+      },
+      "paypal account": {
+        "positive": 20,
+        "negative": 0
+      },
+      "ebay account": {
+        "positive": 5,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "wire transfer": {
+        "positive": 5,
+        "negative": 0
+      },
+      "gift card": {
+        "positive": 0,
+        "negative": 1
+      },
+      "urgent": {
+        "positive": 81,
+        "negative": 48
+      },
+      "urgently": {
+        "positive": 22,
+        "negative": 29
+      },
+      "security alert": {
+        "positive": 2,
+        "negative": 68
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "login to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 71,
+        "negative": 7
+      },
+      "within 24 hours": {
+        "positive": 63,
+        "negative": 2
+      },
+      "within 48 hours": {
+        "positive": 4,
+        "negative": 3
+      }
+    },
+    "label_purity": 0.6106,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 155,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 243
+    },
+    "threeway_accuracy": null,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 0.9522613065326633,
+      "precision": 0.9307692307692308,
+      "recall": 0.9958847736625515,
+      "f1": 0.9622266401590458,
+      "specificity": 0.8838709677419355,
+      "false_positive_rate": 0.11612903225806452,
+      "false_negative_rate": 0.00411522633744856,
+      "confusion_matrix": {
+        "tn": 137,
+        "fp": 18,
+        "fn": 1,
+        "tp": 242
+      },
+      "n": 398,
+      "n_positive": 243,
+      "n_negative": 155,
+      "roc_auc": 0.9961502721359352
+    }
+  },
+  "per_source": {
+    "TREC-06": {
+      "threshold": 0.5,
+      "accuracy": 0.9165746534167587,
+      "precision": 0.7751954821894005,
+      "recall": 0.9165382639958911,
+      "f1": 0.8399623440809603,
+      "specificity": 0.9165860735009671,
+      "false_positive_rate": 0.08341392649903288,
+      "false_negative_rate": 0.08346173600410889,
+      "confusion_matrix": {
+        "tn": 11373,
+        "fp": 1035,
+        "fn": 325,
+        "tp": 3569
+      },
+      "n": 16302,
+      "n_positive": 3894,
+      "n_negative": 12408,
+      "roc_auc": 0.9755163385154697,
+      "label3_counts": {
+        "legit": 12408,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 3894
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 3569,
+        "mean_tokens_full": 280.4,
+        "mean_tokens_fed": 231.1,
+        "median_tokens_full": 201.0,
+        "pct_truncated": 15.0
+      },
+      "tn": {
+        "n": 11373,
+        "mean_tokens_full": 854.8,
+        "mean_tokens_fed": 300.9,
+        "median_tokens_full": 282.0,
+        "pct_truncated": 25.0
+      },
+      "fp": {
+        "n": 1035,
+        "mean_tokens_full": 727.5,
+        "mean_tokens_fed": 213.6,
+        "median_tokens_full": 168.0,
+        "pct_truncated": 12.8
+      },
+      "fn": {
+        "n": 325,
+        "mean_tokens_full": 334.5,
+        "mean_tokens_fed": 241.4,
+        "median_tokens_full": 223.0,
+        "pct_truncated": 17.8
+      },
+      "correct": {
+        "n": 14942,
+        "mean_tokens_full": 717.6,
+        "mean_tokens_fed": 284.3,
+        "median_tokens_full": 254.0,
+        "pct_truncated": 22.7
+      },
+      "wrong": {
+        "n": 1360,
+        "mean_tokens_full": 633.6,
+        "mean_tokens_fed": 220.2,
+        "median_tokens_full": 184.0,
+        "pct_truncated": 14.0
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 1363,
+        "n_positive": 587,
+        "positive_rate": 0.43066764490095377,
+        "recall": 0.899488926746167,
+        "fpr": 0.18170103092783504,
+        "accuracy": 0.8532648569332355
+      },
+      {
+        "bucket": "65-128",
+        "n": 2538,
+        "n_positive": 736,
+        "positive_rate": 0.2899921197793538,
+        "recall": 0.9565217391304348,
+        "fpr": 0.12097669256381798,
+        "accuracy": 0.9014972419227738
+      },
+      {
+        "bucket": "129-256",
+        "n": 4594,
+        "n_positive": 1196,
+        "positive_rate": 0.260339573356552,
+        "recall": 0.8896321070234113,
+        "fpr": 0.10977045320776928,
+        "accuracy": 0.8900740095777101
+      },
+      {
+        "bucket": "257-512",
+        "n": 4232,
+        "n_positive": 780,
+        "positive_rate": 0.1843100189035917,
+        "recall": 0.9435897435897436,
+        "fpr": 0.049536500579374275,
+        "accuracy": 0.9491965973534972
+      },
+      {
+        "bucket": ">512",
+        "n": 3575,
+        "n_positive": 595,
+        "positive_rate": 0.16643356643356644,
+        "recall": 0.9025210084033614,
+        "fpr": 0.04429530201342282,
+        "accuracy": 0.9468531468531468
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 16302,
+    "seconds": 57.73,
+    "rows_per_s": 282.4,
+    "padded_tokens": 4606544,
+    "real_tokens": 4546874,
+    "padded_tokens_per_s": 79791,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 10.06,
+      "p50_ms": 8.92,
+      "p95_ms": 13.84,
+      "max_ms": 18.98,
+      "avg_s": 0.0101,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.136,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 354.05,
+      "p50_ms": 338.14,
+      "p95_ms": 554.84,
+      "max_ms": 606.24,
+      "avg_s": 0.3541,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 4.784,
+      "threads": 16,
+      "estimated_full_run_s": 37.1
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-scratch/best",
+    "run_name": "ft-scratch",
+    "params": 337444867,
+    "timestamp": "2026-10-09 09:57",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-scratch/error_analysis.md",
+    "n_fn": 325,
+    "n_fp": 1035,
+    "composition": {
+      "fp": {
+        "n": 1035,
+        "mailing_list_or_digest": 0.107,
+        "html_heavy": 0.001,
+        "very_short_lt20_tokens": 0.012,
+        "truncated_gt512": 0.128,
+        "lure_phrase": 0.017
+      },
+      "tn": {
+        "n": 11373,
+        "mailing_list_or_digest": 0.31,
+        "html_heavy": 0.002,
+        "very_short_lt20_tokens": 0.005,
+        "truncated_gt512": 0.25,
+        "lure_phrase": 0.012
+      },
+      "fn": {
+        "n": 325,
+        "mailing_list_or_digest": 0.049,
+        "html_heavy": 0.006,
+        "very_short_lt20_tokens": 0.006,
+        "truncated_gt512": 0.178,
+        "lure_phrase": 0.003
+      },
+      "tp": {
+        "n": 3569,
+        "mailing_list_or_digest": 0.028,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.002,
+        "truncated_gt512": 0.15,
+        "lure_phrase": 0.068
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "gci",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "pre-registration",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "nasdaq",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "ringing",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "twx",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "inundated",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "yhoo",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "goog",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "producing",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "targeting",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "fotis",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "e-marketing",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "ibd",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "gannett",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "usages",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "greko's",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 581.21
+      },
+      {
+        "token": "highlight",
+        "share_a": 0.083,
+        "share_b": 0.0,
+        "ratio": 201.05
+      },
+      {
+        "token": "posting",
+        "share_a": 0.083,
+        "share_b": 0.0,
+        "ratio": 201.05
+      },
+      {
+        "token": "vendors",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 193.74
+      },
+      {
+        "token": "dgko",
+        "share_a": 0.08,
+        "share_b": 0.0,
+        "ratio": 193.74
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "contents",
+        "share_a": 0.049,
+        "share_b": 0.012,
+        "ratio": 4.17
+      },
+      {
+        "token": "private",
+        "share_a": 0.054,
+        "share_b": 0.016,
+        "ratio": 3.38
+      },
+      {
+        "token": "dear",
+        "share_a": 0.155,
+        "share_b": 0.062,
+        "ratio": 2.49
+      },
+      {
+        "token": "view",
+        "share_a": 0.052,
+        "share_b": 0.023,
+        "ratio": 2.33
+      },
+      {
+        "token": "become",
+        "share_a": 0.061,
+        "share_b": 0.027,
+        "ratio": 2.25
+      },
+      {
+        "token": "service",
+        "share_a": 0.088,
+        "share_b": 0.046,
+        "ratio": 1.9
+      },
+      {
+        "token": "index",
+        "share_a": 0.059,
+        "share_b": 0.031,
+        "ratio": 1.89
+      },
+      {
+        "token": "website",
+        "share_a": 0.06,
+        "share_b": 0.032,
+        "ratio": 1.86
+      },
+      {
+        "token": "visit",
+        "share_a": 0.058,
+        "share_b": 0.032,
+        "ratio": 1.84
+      },
+      {
+        "token": "battery",
+        "share_a": 0.057,
+        "share_b": 0.033,
+        "ratio": 1.76
+      },
+      {
+        "token": "link",
+        "share_a": 0.066,
+        "share_b": 0.039,
+        "ratio": 1.71
+      },
+      {
+        "token": "free",
+        "share_a": 0.129,
+        "share_b": 0.076,
+        "ratio": 1.7
+      },
+      {
+        "token": "special",
+        "share_a": 0.058,
+        "share_b": 0.035,
+        "ratio": 1.67
+      },
+      {
+        "token": "receive",
+        "share_a": 0.06,
+        "share_b": 0.038,
+        "ratio": 1.61
+      },
+      {
+        "token": "remove",
+        "share_a": 0.053,
+        "share_b": 0.033,
+        "ratio": 1.61
+      },
+      {
+        "token": "world",
+        "share_a": 0.055,
+        "share_b": 0.036,
+        "ratio": 1.53
+      },
+      {
+        "token": "messages",
+        "share_a": 0.079,
+        "share_b": 0.055,
+        "ratio": 1.45
+      },
+      {
+        "token": "anybody",
+        "share_a": 0.063,
+        "share_b": 0.046,
+        "ratio": 1.38
+      },
+      {
+        "token": "open",
+        "share_a": 0.062,
+        "share_b": 0.046,
+        "ratio": 1.35
+      },
+      {
+        "token": "internet",
+        "share_a": 0.062,
+        "share_b": 0.047,
+        "ratio": 1.34
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:052869",
+      "primary:117517",
+      "primary:167537",
+      "primary:006858",
+      "primary:160621",
+      "primary:001821",
+      "primary:186190",
+      "primary:077801",
+      "primary:158787",
+      "primary:150014",
+      "primary:096110",
+      "primary:079715",
+      "primary:060379",
+      "primary:029517",
+      "primary:122099",
+      "primary:161198",
+      "primary:172680",
+      "primary:015259",
+      "primary:022080",
+      "primary:043353",
+      "primary:034854",
+      "primary:055019",
+      "primary:028469",
+      "primary:195636",
+      "primary:178407"
+    ],
+    "top_fp_doc_ids": [
+      "primary:025689",
+      "primary:049322",
+      "primary:066681",
+      "primary:025650",
+      "primary:093533",
+      "primary:118028",
+      "primary:040776",
+      "primary:158244",
+      "primary:197794",
+      "primary:107869",
+      "primary:119762",
+      "primary:142763",
+      "primary:193692",
+      "primary:001010",
+      "primary:064078",
+      "primary:042367",
+      "primary:118388",
+      "primary:121856",
+      "primary:098000",
+      "primary:049912",
+      "primary:057810",
+      "primary:126981",
+      "primary:125508",
+      "primary:138875",
+      "primary:070866"
+    ],
+    "threeway": null
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-scratch/confusion_matrix.png",
+    "roc_curve": "reports/eval_ft-scratch/roc_curve.png",
+    "pr_curve": "reports/eval_ft-scratch/pr_curve.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-scratch/metrics_test3.json b/presentation/reports/eval_ft-scratch/metrics_test3.json
new file mode 100644
index 0000000000000000000000000000000000000000..1bfda8d0bf963d7e9dd3b1d31922ef43265260a7
--- /dev/null
+++ b/presentation/reports/eval_ft-scratch/metrics_test3.json
@@ -0,0 +1,1707 @@
+{
+  "timestamp": "2026-10-09 09:57",
+  "run_name": "ft-scratch",
+  "split": "test3",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout3.parquet",
+  "split_title": "three-way holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-scratch/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-scratch",
+      "init": "scratch",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "cross_entropy",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 0,
+    "best_step": 5009,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9874562778916898,
+    "val_metrics": {
+      "accuracy": 0.9863909971211725,
+      "precision": 0.9863855421686747,
+      "recall": 0.9885293407389519,
+      "f1": 0.9874562778916898,
+      "roc_auc": 0.9984844430897414,
+      "pr_auc": 0.998618660839301,
+      "tn": 6889,
+      "fp": 113,
+      "fn": 95,
+      "tp": 8187,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.6013138890266418,
+      "f1_at_optimal_threshold": 0.9878557186876926,
+      "f1_unwanted": 0.9874562778916898,
+      "n_3way": 8105,
+      "accuracy_3way": 0.9832202344231955,
+      "macro_f1_3way": 0.9537815709229355,
+      "precision_legitimate": 0.9979745370370371,
+      "recall_legitimate": 0.9851471008283348,
+      "f1_legitimate": 0.991519333045853,
+      "precision_phishing": 0.9384164222873901,
+      "recall_phishing": 0.9411764705882353,
+      "f1_phishing": 0.9397944199706314,
+      "precision_spam": 0.8814553990610329,
+      "recall_spam": 0.9842726081258192,
+      "f1_spam": 0.930030959752322,
+      "cm3_ll": 6898,
+      "cm3_lp": 20,
+      "cm3_ls": 84,
+      "cm3_pl": 3,
+      "cm3_pp": 320,
+      "cm3_ps": 17,
+      "cm3_sl": 11,
+      "cm3_sp": 1,
+      "cm3_ss": 751,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 322,
+      "pvs_phish_as_spam": 18,
+      "pvs_spam_as_phish": 1,
+      "pvs_spam_as_spam": 762,
+      "val_loss": 0.04709533672780773,
+      "n": 15284,
+      "eval_seconds": 48.1
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9863909971211725,
+        "precision": 0.9863855421686747,
+        "recall": 0.9885293407389519,
+        "f1": 0.9874562778916898,
+        "roc_auc": 0.9984844430897414,
+        "pr_auc": 0.998618660839301,
+        "tn": 6889,
+        "fp": 113,
+        "fn": 95,
+        "tp": 8187,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.6013138890266418,
+        "f1_at_optimal_threshold": 0.9878557186876926,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.9832202344231955,
+        "macro_f1": 0.9537815709229355,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9979745370370371,
+            "recall": 0.9851471008283348,
+            "f1": 0.991519333045853
+          },
+          "Phishing": {
+            "precision": 0.9384164222873901,
+            "recall": 0.9411764705882353,
+            "f1": 0.9397944199706314
+          },
+          "Spam": {
+            "precision": 0.8814553990610329,
+            "recall": 0.9842726081258192,
+            "f1": 0.930030959752322
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6898,
+              20,
+              84
+            ],
+            [
+              3,
+              320,
+              17
+            ],
+            [
+              11,
+              1,
+              751
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              322,
+              18
+            ],
+            [
+              1,
+              762
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.6013138890266418,
+      "val_f1_at_optimal": 0.9878557186876926
+    },
+    "init": "scratch",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 3220,
+    "positive": 1221,
+    "negative": 1999,
+    "positive_rate": 0.3792,
+    "label3": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "by_source": {
+      "Assassin": {
+        "rows": 236,
+        "positive": 72,
+        "label3": {
+          "legit": 164,
+          "phish": 0,
+          "spam": 72,
+          "unwanted": 0
+        }
+      },
+      "CEAS-08": {
+        "rows": 736,
+        "positive": 0,
+        "label3": {
+          "legit": 736,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Enron": {
+        "rows": 628,
+        "positive": 628,
+        "label3": {
+          "legit": 0,
+          "phish": 0,
+          "spam": 628,
+          "unwanted": 0
+        }
+      },
+      "Nazario": {
+        "rows": 222,
+        "positive": 222,
+        "label3": {
+          "legit": 0,
+          "phish": 222,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Nigerian_Fraud": {
+        "rows": 299,
+        "positive": 299,
+        "label3": {
+          "legit": 0,
+          "phish": 299,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "TREC-07": {
+        "rows": 1099,
+        "positive": 0,
+        "label3": {
+          "legit": 1099,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      }
+    },
+    "mean_tokens_full": 835.2,
+    "pct_truncated": 34.01
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9822981366459628,
+    "precision": 0.9762684124386252,
+    "recall": 0.9770679770679771,
+    "f1": 0.9766680311092918,
+    "specificity": 0.9854927463731866,
+    "false_positive_rate": 0.014507253626813406,
+    "false_negative_rate": 0.02293202293202293,
+    "confusion_matrix": {
+      "tn": 1970,
+      "fp": 29,
+      "fn": 28,
+      "tp": 1193
+    },
+    "n": 3220,
+    "n_positive": 1221,
+    "n_negative": 1999
+  },
+  "threshold_free": {
+    "roc_auc": 0.9983431519199404,
+    "pr_auc": 0.9974248605846625,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.9672399672399672,
+      "fpr": 0.0080040020010005,
+      "threshold": 0.7805293202400208,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.8353808353808354,
+      "fpr": 0.0005002501250625312,
+      "threshold": 0.9868443012237549,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.365547776222229,
+      "f1": 0.9775601795185639,
+      "precision": 0.9739837398373984,
+      "recall": 0.9811629811629812
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 1193,
+        "predicted_phishing": 485,
+        "predicted_spam": 708
+      },
+      "false_positives_flagged": {
+        "n": 29,
+        "predicted_phishing": 3,
+        "predicted_spam": 26
+      },
+      "missed_positives": {
+        "n": 28,
+        "predicted_phishing": 7,
+        "predicted_spam": 21
+      },
+      "all_flagged": {
+        "n": 1222,
+        "predicted_phishing": 488,
+        "predicted_spam": 734
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 0,
+        "predicted_phishing": 0,
+        "predicted_spam": 0
+      }
+    },
+    "rows_with_full_label": 3220,
+    "rows_unwanted_only": 0,
+    "evaluated": true,
+    "reason": null,
+    "confusion_matrix": {
+      "labels": [
+        "Legitimate",
+        "Phishing",
+        "Spam"
+      ],
+      "matrix": [
+        [
+          1970,
+          3,
+          26
+        ],
+        [
+          9,
+          483,
+          29
+        ],
+        [
+          19,
+          2,
+          679
+        ]
+      ],
+      "cells": {
+        "true_legitimate": {
+          "pred_legitimate": 1970,
+          "pred_phishing": 3,
+          "pred_spam": 26
+        },
+        "true_phishing": {
+          "pred_legitimate": 9,
+          "pred_phishing": 483,
+          "pred_spam": 29
+        },
+        "true_spam": {
+          "pred_legitimate": 19,
+          "pred_phishing": 2,
+          "pred_spam": 679
+        }
+      }
+    },
+    "accuracy": 0.9726708074534162,
+    "macro_f1": 0.9633747490819639,
+    "macro_f1_present_classes": 0.9633747490819639,
+    "classes_present": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "per_class": {
+      "Legitimate": {
+        "precision": 0.985985985985986,
+        "recall": 0.9854927463731866,
+        "f1": 0.9857393044783588,
+        "support": 1999
+      },
+      "Phishing": {
+        "precision": 0.9897540983606558,
+        "recall": 0.927063339731286,
+        "f1": 0.9573835480673935,
+        "support": 521
+      },
+      "Spam": {
+        "precision": 0.9250681198910081,
+        "recall": 0.97,
+        "f1": 0.9470013947001394,
+        "support": 700
+      }
+    },
+    "argmax_rule": {
+      "doc": "plain argmax over the three probabilities (secondary; can disagree with the binary verdict near the threshold)",
+      "accuracy": 0.9729813664596273,
+      "macro_f1": 0.9636795971999454,
+      "per_class": {
+        "Legitimate": {
+          "precision": 0.9859929964982491,
+          "recall": 0.9859929964982491,
+          "f1": 0.9859929964982491,
+          "support": 1999
+        },
+        "Phishing": {
+          "precision": 0.9897540983606558,
+          "recall": 0.927063339731286,
+          "f1": 0.9573835480673935,
+          "support": 521
+        },
+        "Spam": {
+          "precision": 0.9263301500682128,
+          "recall": 0.97,
+          "f1": 0.947662247034194,
+          "support": 700
+        }
+      },
+      "rows_where_rules_differ": 1
+    },
+    "phish_vs_spam_on_true_unwanted": {
+      "n_true_unwanted": 1221,
+      "kind_only": {
+        "doc": "given the row is unwanted, is argmax(P(phishing), P(spam)) right? (independent of the roll-up threshold)",
+        "labels": [
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            489,
+            32
+          ],
+          [
+            3,
+            697
+          ]
+        ],
+        "cells": {
+          "true_phishing": {
+            "pred_phishing": 489,
+            "pred_spam": 32
+          },
+          "true_spam": {
+            "pred_phishing": 3,
+            "pred_spam": 697
+          }
+        },
+        "accuracy": 0.9713349713349714,
+        "phishing_as_spam": 32,
+        "spam_as_phishing": 3
+      },
+      "rollup_rule": {
+        "doc": "rows = true Phishing / Spam; columns = predicted Legitimate (missed) / Phishing / Spam under the roll-up rule",
+        "rows": [
+          "Phishing",
+          "Spam"
+        ],
+        "columns": [
+          "Legitimate",
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            9,
+            483,
+            29
+          ],
+          [
+            19,
+            2,
+            679
+          ]
+        ],
+        "missed_as_legitimate": {
+          "phishing": 9,
+          "spam": 19
+        }
+      }
+    },
+    "per_source": {
+      "Assassin": {
+        "n": 236,
+        "counts": {
+          "Legitimate": 164,
+          "Phishing": 0,
+          "Spam": 72
+        },
+        "accuracy": 0.940677966101695,
+        "macro_f1": 0.6219893206089525,
+        "macro_f1_present_classes": 0.9329839809134288,
+        "per_class_recall": {
+          "Legitimate": 0.9573170731707317,
+          "Spam": 0.9027777777777778
+        }
+      },
+      "CEAS-08": {
+        "n": 736,
+        "counts": {
+          "Legitimate": 736,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9796195652173914,
+        "macro_f1": 0.3299016243422558,
+        "macro_f1_present_classes": 0.9897048730267674,
+        "per_class_recall": {
+          "Legitimate": 0.9796195652173914
+        }
+      },
+      "Enron": {
+        "n": 628,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 0,
+          "Spam": 628
+        },
+        "accuracy": 0.9777070063694268,
+        "macro_f1": 0.3295759527643585,
+        "macro_f1_present_classes": 0.9887278582930756,
+        "per_class_recall": {
+          "Spam": 0.9777070063694268
+        }
+      },
+      "Nazario": {
+        "n": 222,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 222,
+          "Spam": 0
+        },
+        "accuracy": 0.8963963963963963,
+        "macro_f1": 0.31512272367379257,
+        "macro_f1_present_classes": 0.9453681710213777,
+        "per_class_recall": {
+          "Phishing": 0.8963963963963963
+        }
+      },
+      "Nigerian_Fraud": {
+        "n": 299,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 299,
+          "Spam": 0
+        },
+        "accuracy": 0.9498327759197325,
+        "macro_f1": 0.324757004002287,
+        "macro_f1_present_classes": 0.9742710120068611,
+        "per_class_recall": {
+          "Phishing": 0.9498327759197325
+        }
+      },
+      "TREC-07": {
+        "n": 1099,
+        "counts": {
+          "Legitimate": 1099,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9936305732484076,
+        "macro_f1": 0.33226837060702874,
+        "macro_f1_present_classes": 0.9968051118210862,
+        "per_class_recall": {
+          "Legitimate": 0.9936305732484076
+        }
+      }
+    }
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 139,
+      "bank account": 91,
+      "urgently": 25,
+      "verify your account": 15,
+      "confirm your account": 9,
+      "update your account": 9,
+      "within 24 hours": 8,
+      "security alert": 8,
+      "paypal account": 7,
+      "password will expire": 7,
+      "click here to verify": 6,
+      "unusual activity": 4,
+      "wire transfer": 3,
+      "update your information": 3,
+      "gift card": 3,
+      "verify your identity": 2,
+      "unusual sign-in": 2,
+      "account will be suspended": 2,
+      "update your payment": 2,
+      "failure to verify": 2,
+      "within 48 hours": 2,
+      "confirm your identity": 2,
+      "enter your password": 1,
+      "update your billing": 1,
+      "reset your password": 1,
+      "suspicious activity": 1,
+      "security notice": 1,
+      "log in to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 15,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "update your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 2,
+        "negative": 1
+      },
+      "update your billing": {
+        "positive": 1,
+        "negative": 0
+      },
+      "update your payment": {
+        "positive": 2,
+        "negative": 0
+      },
+      "click here to verify": {
+        "positive": 6,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual sign-in": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "suspicious activity": {
+        "positive": 1,
+        "negative": 0
+      },
+      "paypal account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "password will expire": {
+        "positive": 7,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 1,
+        "negative": 0
+      },
+      "wire transfer": {
+        "positive": 2,
+        "negative": 1
+      },
+      "gift card": {
+        "positive": 3,
+        "negative": 0
+      },
+      "urgent": {
+        "positive": 133,
+        "negative": 6
+      },
+      "urgently": {
+        "positive": 25,
+        "negative": 0
+      },
+      "security alert": {
+        "positive": 8,
+        "negative": 0
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 91,
+        "negative": 0
+      },
+      "failure to verify": {
+        "positive": 2,
+        "negative": 0
+      },
+      "within 24 hours": {
+        "positive": 8,
+        "negative": 0
+      },
+      "within 48 hours": {
+        "positive": 2,
+        "negative": 0
+      }
+    },
+    "label_purity": 0.9745,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 7,
+      "phish": 236,
+      "spam": 32,
+      "unwanted": 0
+    },
+    "threeway_accuracy": 0.9563636363636364,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 0.9927272727272727,
+      "precision": 1.0,
+      "recall": 0.9925373134328358,
+      "f1": 0.9962546816479401,
+      "specificity": 1.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.007462686567164179,
+      "confusion_matrix": {
+        "tn": 7,
+        "fp": 0,
+        "fn": 2,
+        "tp": 266
+      },
+      "n": 275,
+      "n_positive": 268,
+      "n_negative": 7,
+      "roc_auc": 1.0
+    }
+  },
+  "per_source": {
+    "Assassin": {
+      "threshold": 0.5,
+      "accuracy": 0.9491525423728814,
+      "precision": 0.9054054054054054,
+      "recall": 0.9305555555555556,
+      "f1": 0.9178082191780821,
+      "specificity": 0.9573170731707317,
+      "false_positive_rate": 0.042682926829268296,
+      "false_negative_rate": 0.06944444444444445,
+      "confusion_matrix": {
+        "tn": 157,
+        "fp": 7,
+        "fn": 5,
+        "tp": 67
+      },
+      "n": 236,
+      "n_positive": 72,
+      "n_negative": 164,
+      "roc_auc": 0.9881436314363145,
+      "label3_counts": {
+        "legit": 164,
+        "phish": 0,
+        "spam": 72,
+        "unwanted": 0
+      }
+    },
+    "CEAS-08": {
+      "threshold": 0.5,
+      "accuracy": 0.9796195652173914,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9796195652173914,
+      "false_positive_rate": 0.020380434782608696,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 721,
+        "fp": 15,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 736,
+      "n_positive": 0,
+      "n_negative": 736,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 736,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Enron": {
+      "threshold": 0.5,
+      "accuracy": 0.9777070063694268,
+      "precision": 1.0,
+      "recall": 0.9777070063694268,
+      "f1": 0.9887278582930756,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.022292993630573247,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 14,
+        "tp": 614
+      },
+      "n": 628,
+      "n_positive": 628,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 0,
+        "spam": 628,
+        "unwanted": 0
+      }
+    },
+    "Nazario": {
+      "threshold": 0.5,
+      "accuracy": 0.9684684684684685,
+      "precision": 1.0,
+      "recall": 0.9684684684684685,
+      "f1": 0.9839816933638443,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.03153153153153153,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 7,
+        "tp": 215
+      },
+      "n": 222,
+      "n_positive": 222,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 222,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Nigerian_Fraud": {
+      "threshold": 0.5,
+      "accuracy": 0.9933110367892977,
+      "precision": 1.0,
+      "recall": 0.9933110367892977,
+      "f1": 0.9966442953020134,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.006688963210702341,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 2,
+        "tp": 297
+      },
+      "n": 299,
+      "n_positive": 299,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 299,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "TREC-07": {
+      "threshold": 0.5,
+      "accuracy": 0.9936305732484076,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9936305732484076,
+      "false_positive_rate": 0.006369426751592357,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 1092,
+        "fp": 7,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 1099,
+      "n_positive": 0,
+      "n_negative": 1099,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 1099,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 1193,
+        "mean_tokens_full": 614.9,
+        "mean_tokens_fed": 279.6,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.4
+      },
+      "tn": {
+        "n": 1970,
+        "mean_tokens_full": 733.2,
+        "mean_tokens_fed": 358.4,
+        "median_tokens_full": 392.5,
+        "pct_truncated": 38.3
+      },
+      "fp": {
+        "n": 29,
+        "mean_tokens_full": 1182.1,
+        "mean_tokens_fed": 368.0,
+        "median_tokens_full": 462.0,
+        "pct_truncated": 48.3
+      },
+      "fn": {
+        "n": 28,
+        "mean_tokens_full": 17044.3,
+        "mean_tokens_fed": 291.4,
+        "median_tokens_full": 291.0,
+        "pct_truncated": 39.3
+      },
+      "correct": {
+        "n": 3163,
+        "mean_tokens_full": 688.5,
+        "mean_tokens_fed": 328.7,
+        "median_tokens_full": 338.0,
+        "pct_truncated": 33.8
+      },
+      "wrong": {
+        "n": 57,
+        "mean_tokens_full": 8974.1,
+        "mean_tokens_fed": 330.4,
+        "median_tokens_full": 368.0,
+        "pct_truncated": 43.9
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "unwanted": {
+        "n": 1221,
+        "mean_tokens_full": 991.6,
+        "mean_tokens_fed": 279.9,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.7
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "phish": {
+        "n": 521,
+        "mean_tokens_full": 1349.4,
+        "mean_tokens_fed": 355.1,
+        "median_tokens_full": 412.0,
+        "pct_truncated": 39.9
+      },
+      "spam": {
+        "n": 700,
+        "mean_tokens_full": 725.3,
+        "mean_tokens_fed": 223.9,
+        "median_tokens_full": 159.0,
+        "pct_truncated": 16.9
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 129,
+        "n_positive": 99,
+        "positive_rate": 0.7674418604651163,
+        "recall": 0.9191919191919192,
+        "fpr": 0.06666666666666667,
+        "accuracy": 0.9224806201550387
+      },
+      {
+        "bucket": "65-128",
+        "n": 427,
+        "n_positive": 252,
+        "positive_rate": 0.5901639344262295,
+        "recall": 0.9920634920634921,
+        "fpr": 0.011428571428571429,
+        "accuracy": 0.990632318501171
+      },
+      {
+        "bucket": "129-256",
+        "n": 699,
+        "n_positive": 276,
+        "positive_rate": 0.3948497854077253,
+        "recall": 0.9891304347826086,
+        "fpr": 0.01182033096926714,
+        "accuracy": 0.9885550786838341
+      },
+      {
+        "bucket": "257-512",
+        "n": 870,
+        "n_positive": 268,
+        "positive_rate": 0.3080459770114943,
+        "recall": 0.9850746268656716,
+        "fpr": 0.009966777408637873,
+        "accuracy": 0.9885057471264368
+      },
+      {
+        "bucket": ">512",
+        "n": 1095,
+        "n_positive": 326,
+        "positive_rate": 0.29771689497716897,
+        "recall": 0.9662576687116564,
+        "fpr": 0.018205461638491547,
+        "accuracy": 0.9771689497716894
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 3220,
+    "seconds": 13.56,
+    "rows_per_s": 237.5,
+    "padded_tokens": 1079936,
+    "real_tokens": 1058490,
+    "padded_tokens_per_s": 79640,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 10.06,
+      "p50_ms": 8.92,
+      "p95_ms": 13.84,
+      "max_ms": 18.98,
+      "avg_s": 0.0101,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.136,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 354.05,
+      "p50_ms": 338.14,
+      "p95_ms": 554.84,
+      "max_ms": 606.24,
+      "avg_s": 0.3541,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 4.784,
+      "threads": 16,
+      "estimated_full_run_s": 37.1
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-scratch/best",
+    "run_name": "ft-scratch",
+    "params": 337444867,
+    "timestamp": "2026-10-09 09:57",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-scratch/error_analysis_test3.md",
+    "n_fn": 28,
+    "n_fp": 29,
+    "composition": {
+      "fp": {
+        "n": 29,
+        "mailing_list_or_digest": 0.586,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.483,
+        "lure_phrase": 0.0
+      },
+      "tn": {
+        "n": 1970,
+        "mailing_list_or_digest": 0.562,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.383,
+        "lure_phrase": 0.004
+      },
+      "fn": {
+        "n": 28,
+        "mailing_list_or_digest": 0.25,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.107,
+        "truncated_gt512": 0.393,
+        "lure_phrase": 0.071
+      },
+      "tp": {
+        "n": 1193,
+        "mailing_list_or_digest": 0.083,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.005,
+        "truncated_gt512": 0.264,
+        "lure_phrase": 0.223
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "hoovers",
+        "share_a": 0.107,
+        "share_b": 0.0,
+        "ratio": 293.14
+      },
+      {
+        "token": "jason",
+        "share_a": 0.107,
+        "share_b": 0.0,
+        "ratio": 293.14
+      },
+      {
+        "token": "columbia",
+        "share_a": 0.107,
+        "share_b": 0.0,
+        "ratio": 293.14
+      },
+      {
+        "token": "chin",
+        "share_a": 0.107,
+        "share_b": 0.001,
+        "ratio": 97.71
+      },
+      {
+        "token": "simpson",
+        "share_a": 0.107,
+        "share_b": 0.001,
+        "ratio": 97.71
+      },
+      {
+        "token": "joe",
+        "share_a": 0.107,
+        "share_b": 0.001,
+        "ratio": 97.71
+      },
+      {
+        "token": "miami",
+        "share_a": 0.107,
+        "share_b": 0.001,
+        "ratio": 97.71
+      },
+      {
+        "token": "jennifer",
+        "share_a": 0.107,
+        "share_b": 0.001,
+        "ratio": 97.71
+      },
+      {
+        "token": "streams",
+        "share_a": 0.143,
+        "share_b": 0.002,
+        "ratio": 75.38
+      },
+      {
+        "token": "publishing",
+        "share_a": 0.107,
+        "share_b": 0.002,
+        "ratio": 58.63
+      },
+      {
+        "token": "cut",
+        "share_a": 0.107,
+        "share_b": 0.002,
+        "ratio": 58.63
+      },
+      {
+        "token": "economy",
+        "share_a": 0.107,
+        "share_b": 0.002,
+        "ratio": 58.63
+      },
+      {
+        "token": "zzzz",
+        "share_a": 0.143,
+        "share_b": 0.003,
+        "ratio": 53.84
+      },
+      {
+        "token": "invite",
+        "share_a": 0.143,
+        "share_b": 0.003,
+        "ratio": 53.84
+      },
+      {
+        "token": "event",
+        "share_a": 0.179,
+        "share_b": 0.004,
+        "ratio": 41.88
+      },
+      {
+        "token": "club",
+        "share_a": 0.143,
+        "share_b": 0.003,
+        "ratio": 41.88
+      },
+      {
+        "token": "tables",
+        "share_a": 0.107,
+        "share_b": 0.003,
+        "ratio": 41.88
+      },
+      {
+        "token": "comments",
+        "share_a": 0.107,
+        "share_b": 0.003,
+        "ratio": 41.88
+      },
+      {
+        "token": "environment",
+        "share_a": 0.143,
+        "share_b": 0.004,
+        "ratio": 34.26
+      },
+      {
+        "token": "url",
+        "share_a": 0.143,
+        "share_b": 0.004,
+        "ratio": 34.26
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "promotional",
+        "share_a": 0.138,
+        "share_b": 0.001,
+        "ratio": 200.39
+      },
+      {
+        "token": "cook",
+        "share_a": 0.103,
+        "share_b": 0.001,
+        "ratio": 93.52
+      },
+      {
+        "token": "mill",
+        "share_a": 0.103,
+        "share_b": 0.001,
+        "ratio": 93.52
+      },
+      {
+        "token": "palm",
+        "share_a": 0.103,
+        "share_b": 0.002,
+        "ratio": 66.8
+      },
+      {
+        "token": "full-color",
+        "share_a": 0.138,
+        "share_b": 0.003,
+        "ratio": 54.65
+      },
+      {
+        "token": "rock",
+        "share_a": 0.103,
+        "share_b": 0.003,
+        "ratio": 35.97
+      },
+      {
+        "token": "gift",
+        "share_a": 0.138,
+        "share_b": 0.004,
+        "ratio": 35.36
+      },
+      {
+        "token": "frequently",
+        "share_a": 0.103,
+        "share_b": 0.004,
+        "ratio": 31.17
+      },
+      {
+        "token": "funny",
+        "share_a": 0.103,
+        "share_b": 0.004,
+        "ratio": 31.17
+      },
+      {
+        "token": "editors",
+        "share_a": 0.103,
+        "share_b": 0.004,
+        "ratio": 31.17
+      },
+      {
+        "token": "classic",
+        "share_a": 0.103,
+        "share_b": 0.004,
+        "ratio": 27.5
+      },
+      {
+        "token": "multi-part",
+        "share_a": 0.138,
+        "share_b": 0.006,
+        "ratio": 26.14
+      },
+      {
+        "token": "draw",
+        "share_a": 0.103,
+        "share_b": 0.005,
+        "ratio": 24.61
+      },
+      {
+        "token": "viewing",
+        "share_a": 0.138,
+        "share_b": 0.006,
+        "ratio": 24.05
+      },
+      {
+        "token": "powered",
+        "share_a": 0.138,
+        "share_b": 0.006,
+        "ratio": 24.05
+      },
+      {
+        "token": "deals",
+        "share_a": 0.103,
+        "share_b": 0.005,
+        "ratio": 22.27
+      },
+      {
+        "token": "serves",
+        "share_a": 0.103,
+        "share_b": 0.005,
+        "ratio": 22.27
+      },
+      {
+        "token": "guaranteed",
+        "share_a": 0.103,
+        "share_b": 0.005,
+        "ratio": 22.27
+      },
+      {
+        "token": "content-transfer-encoding",
+        "share_a": 0.138,
+        "share_b": 0.007,
+        "ratio": 20.73
+      },
+      {
+        "token": "content-disposition",
+        "share_a": 0.103,
+        "share_b": 0.006,
+        "ratio": 20.33
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:148887",
+      "aura:Nazario:000097",
+      "primary:181246",
+      "primary:025244",
+      "aura:Nazario:000551",
+      "primary:148857",
+      "primary:173930",
+      "aura:Nazario:000643",
+      "primary:007176",
+      "aura:Nazario:000533",
+      "primary:001939",
+      "aura:Nazario:001540",
+      "primary:129677",
+      "primary:135631",
+      "aura:Nigerian_Fraud:001032",
+      "primary:046569",
+      "primary:014300",
+      "aura:Nazario:000299",
+      "primary:181468",
+      "primary:130516",
+      "primary:164494",
+      "primary:201972",
+      "aura:Nazario:001468",
+      "primary:157116",
+      "primary:048857"
+    ],
+    "top_fp_doc_ids": [
+      "primary:156887",
+      "primary:118434",
+      "primary:188362",
+      "primary:066021",
+      "primary:199386",
+      "primary:048756",
+      "primary:109226",
+      "primary:130385",
+      "primary:027029",
+      "primary:144048",
+      "primary:200749",
+      "primary:057514",
+      "primary:152763",
+      "primary:025142",
+      "primary:153841",
+      "primary:165228",
+      "primary:024251",
+      "primary:101377",
+      "primary:021911",
+      "primary:061178",
+      "primary:012585",
+      "primary:128544",
+      "primary:103927",
+      "primary:159406",
+      "primary:059404"
+    ],
+    "threeway": {
+      "n_phish_as_spam": 29,
+      "n_spam_as_phish": 2,
+      "top_phish_as_spam_doc_ids": [
+        "aura:Nazario:000401",
+        "aura:Nazario:000832",
+        "aura:Nigerian_Fraud:000262",
+        "aura:Nigerian_Fraud:000824",
+        "aura:Nazario:000328",
+        "aura:Nigerian_Fraud:003000",
+        "aura:Nigerian_Fraud:000266",
+        "aura:Nigerian_Fraud:000337",
+        "aura:Nazario:000946",
+        "aura:Nigerian_Fraud:003049",
+        "aura:Nazario:000105",
+        "aura:Nazario:001179",
+        "aura:Nazario:001170",
+        "aura:Nazario:000214",
+        "aura:Nazario:000879",
+        "aura:Nazario:000104",
+        "aura:Nazario:001290",
+        "aura:Nigerian_Fraud:001100",
+        "aura:Nigerian_Fraud:000101",
+        "aura:Nazario:001287",
+        "aura:Nigerian_Fraud:001783",
+        "aura:Nigerian_Fraud:002955",
+        "aura:Nazario:000020",
+        "aura:Nazario:000426",
+        "aura:Nigerian_Fraud:001698"
+      ],
+      "top_spam_as_phish_doc_ids": [
+        "primary:077835",
+        "primary:197304"
+      ],
+      "phish_as_spam_tokens": [
+        {
+          "token": "whatsapp",
+          "share_a": 0.103,
+          "share_b": 0.0,
+          "ratio": 114.73
+        },
+        {
+          "token": "nova",
+          "share_a": 0.103,
+          "share_b": 0.0,
+          "ratio": 114.73
+        },
+        {
+          "token": "abra",
+          "share_a": 0.103,
+          "share_b": 0.0,
+          "ratio": 114.73
+        },
+        {
+          "token": "content-type",
+          "share_a": 0.103,
+          "share_b": 0.0,
+          "ratio": 114.73
+        },
+        {
+          "token": "content-transfer-encoding",
+          "share_a": 0.103,
+          "share_b": 0.0,
+          "ratio": 114.73
+        },
+        {
+          "token": "para",
+          "share_a": 0.138,
+          "share_b": 0.002,
+          "ratio": 49.17
+        },
+        {
+          "token": "removed",
+          "share_a": 0.172,
+          "share_b": 0.004,
+          "ratio": 36.06
+        },
+        {
+          "token": "income",
+          "share_a": 0.138,
+          "share_b": 0.006,
+          "ratio": 21.07
+        },
+        {
+          "token": "los",
+          "share_a": 0.103,
+          "share_b": 0.006,
+          "ratio": 16.39
+        },
+        {
+          "token": "building",
+          "share_a": 0.103,
+          "share_b": 0.008,
+          "ratio": 12.75
+        },
+        {
+          "token": "plain",
+          "share_a": 0.138,
+          "share_b": 0.012,
+          "ratio": 11.35
+        },
+        {
+          "token": "disclaimer",
+          "share_a": 0.103,
+          "share_b": 0.01,
+          "ratio": 10.43
+        },
+        {
+          "token": "bit",
+          "share_a": 0.103,
+          "share_b": 0.012,
+          "ratio": 8.83
+        },
+        {
+          "token": "fill",
+          "share_a": 0.103,
+          "share_b": 0.012,
+          "ratio": 8.83
+        },
+        {
+          "token": "plus",
+          "share_a": 0.103,
+          "share_b": 0.012,
+          "ratio": 8.83
+        },
+        {
+          "token": "register",
+          "share_a": 0.103,
+          "share_b": 0.012,
+          "ratio": 8.83
+        },
+        {
+          "token": "return",
+          "share_a": 0.138,
+          "share_b": 0.017,
+          "ratio": 8.68
+        },
+        {
+          "token": "text",
+          "share_a": 0.138,
+          "share_b": 0.017,
+          "ratio": 8.68
+        },
+        {
+          "token": "plan",
+          "share_a": 0.138,
+          "share_b": 0.019,
+          "ratio": 7.76
+        },
+        {
+          "token": "links",
+          "share_a": 0.103,
+          "share_b": 0.014,
+          "ratio": 7.65
+        }
+      ],
+      "spam_as_phish_tokens": []
+    }
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-scratch/confusion_matrix_test3.png",
+    "confusion_matrix3": "reports/eval_ft-scratch/confusion_matrix3_test3.png",
+    "roc_curve": "reports/eval_ft-scratch/roc_curve_test3.png",
+    "pr_curve": "reports/eval_ft-scratch/pr_curve_test3.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-synthetic/metrics.json b/presentation/reports/eval_ft-synthetic/metrics.json
new file mode 100644
index 0000000000000000000000000000000000000000..5f580b1677b350a60bc4426d7f88c217c38cc07c
--- /dev/null
+++ b/presentation/reports/eval_ft-synthetic/metrics.json
@@ -0,0 +1,1109 @@
+{
+  "timestamp": "2026-10-09 08:16",
+  "run_name": "ft-synthetic",
+  "split": "test",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet",
+  "split_title": "TREC-06 binary holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-synthetic/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-synthetic",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "cross_entropy",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "on",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 0,
+    "best_step": 5000,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9926514877725575,
+    "val_metrics": {
+      "accuracy": 0.9920177963883801,
+      "precision": 0.9903846153846154,
+      "recall": 0.9949287611687998,
+      "f1": 0.9926514877725575,
+      "roc_auc": 0.9996055651398734,
+      "pr_auc": 0.9996662445428656,
+      "tn": 6922,
+      "fp": 80,
+      "fn": 42,
+      "tp": 8240,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.6093423366546631,
+      "f1_at_optimal_threshold": 0.9930643507629214,
+      "f1_unwanted": 0.9926514877725575,
+      "n_3way": 8105,
+      "accuracy_3way": 0.9891425046267736,
+      "macro_f1_3way": 0.9665378162175431,
+      "precision_legitimate": 0.9992783951508154,
+      "recall_legitimate": 0.988860325621251,
+      "f1_legitimate": 0.9940420644605555,
+      "precision_phishing": 0.8805194805194805,
+      "recall_phishing": 0.9970588235294118,
+      "f1_phishing": 0.9351724137931035,
+      "precision_spam": 0.9532237673830595,
+      "recall_spam": 0.9882044560943644,
+      "f1_spam": 0.9703989703989704,
+      "cm3_ll": 6924,
+      "cm3_lp": 41,
+      "cm3_ls": 37,
+      "cm3_pl": 1,
+      "cm3_pp": 339,
+      "cm3_ps": 0,
+      "cm3_sl": 4,
+      "cm3_sp": 5,
+      "cm3_ss": 754,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 340,
+      "pvs_phish_as_spam": 0,
+      "pvs_spam_as_phish": 6,
+      "pvs_spam_as_spam": 757,
+      "val_loss": 0.027758434301769195,
+      "n": 15284,
+      "eval_seconds": 47.8
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9920177963883801,
+        "precision": 0.9903846153846154,
+        "recall": 0.9949287611687998,
+        "f1": 0.9926514877725575,
+        "roc_auc": 0.9996055651398734,
+        "pr_auc": 0.9996662445428656,
+        "tn": 6922,
+        "fp": 80,
+        "fn": 42,
+        "tp": 8240,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.6093423366546631,
+        "f1_at_optimal_threshold": 0.9930643507629214,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.9891425046267736,
+        "macro_f1": 0.9665378162175431,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9992783951508154,
+            "recall": 0.988860325621251,
+            "f1": 0.9940420644605555
+          },
+          "Phishing": {
+            "precision": 0.8805194805194805,
+            "recall": 0.9970588235294118,
+            "f1": 0.9351724137931035
+          },
+          "Spam": {
+            "precision": 0.9532237673830595,
+            "recall": 0.9882044560943644,
+            "f1": 0.9703989703989704
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6924,
+              41,
+              37
+            ],
+            [
+              1,
+              339,
+              0
+            ],
+            [
+              4,
+              5,
+              754
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              340,
+              0
+            ],
+            [
+              6,
+              757
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.6093423366546631,
+      "val_f1_at_optimal": 0.9930643507629214
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 16302,
+    "positive": 3894,
+    "negative": 12408,
+    "positive_rate": 0.2389,
+    "label3": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "by_source": {
+      "TREC-06": {
+        "rows": 16302,
+        "positive": 3894,
+        "label3": {
+          "legit": 12408,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 3894
+        }
+      }
+    },
+    "mean_tokens_full": 710.6,
+    "pct_truncated": 21.93
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9483498957183167,
+    "precision": 0.8401694159607669,
+    "recall": 0.9678993323061119,
+    "f1": 0.8995226730310263,
+    "specificity": 0.9422147001934236,
+    "false_positive_rate": 0.0577852998065764,
+    "false_negative_rate": 0.03210066769388803,
+    "confusion_matrix": {
+      "tn": 11691,
+      "fp": 717,
+      "fn": 125,
+      "tp": 3769
+    },
+    "n": 16302,
+    "n_positive": 3894,
+    "n_negative": 12408
+  },
+  "threshold_free": {
+    "roc_auc": 0.990803500202166,
+    "pr_auc": 0.971579198733221,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.7871083718541345,
+      "fpr": 0.009993552546744036,
+      "threshold": 0.9979060292243958,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.47791474062660505,
+      "fpr": 0.0008865248226950354,
+      "threshold": 0.9998263120651245,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.9621819853782654,
+      "f1": 0.9185659411011523,
+      "precision": 0.9159856996935649,
+      "recall": 0.921160760143811
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 3769,
+        "predicted_phishing": 1486,
+        "predicted_spam": 2283
+      },
+      "false_positives_flagged": {
+        "n": 717,
+        "predicted_phishing": 534,
+        "predicted_spam": 183
+      },
+      "missed_positives": {
+        "n": 125,
+        "predicted_phishing": 76,
+        "predicted_spam": 49
+      },
+      "all_flagged": {
+        "n": 4486,
+        "predicted_phishing": 2020,
+        "predicted_spam": 2466
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 3769,
+        "predicted_phishing": 1486,
+        "predicted_spam": 2283
+      }
+    },
+    "rows_with_full_label": 12408,
+    "rows_unwanted_only": 3894,
+    "evaluated": false,
+    "reason": "no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)"
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 129,
+      "bank account": 78,
+      "security alert": 70,
+      "within 24 hours": 65,
+      "urgently": 51,
+      "paypal account": 20,
+      "within 48 hours": 7,
+      "verify your account": 7,
+      "update your account": 6,
+      "account will be suspended": 5,
+      "ebay account": 5,
+      "enter your password": 5,
+      "wire transfer": 5,
+      "unusual activity": 5,
+      "confirm your identity": 4,
+      "verify your identity": 4,
+      "click here to verify": 3,
+      "log in to your account": 2,
+      "confirm your account": 2,
+      "security notice": 1,
+      "gift card": 1,
+      "reset your password": 1,
+      "account has been locked": 1,
+      "update your information": 1,
+      "account has been limited": 1,
+      "login to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 5,
+        "negative": 0
+      },
+      "update your account": {
+        "positive": 6,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 0,
+        "negative": 1
+      },
+      "click here to verify": {
+        "positive": 3,
+        "negative": 0
+      },
+      "account has been limited": {
+        "positive": 1,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 4,
+        "negative": 1
+      },
+      "account has been locked": {
+        "positive": 1,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 2,
+        "negative": 3
+      },
+      "paypal account": {
+        "positive": 20,
+        "negative": 0
+      },
+      "ebay account": {
+        "positive": 5,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "wire transfer": {
+        "positive": 5,
+        "negative": 0
+      },
+      "gift card": {
+        "positive": 0,
+        "negative": 1
+      },
+      "urgent": {
+        "positive": 81,
+        "negative": 48
+      },
+      "urgently": {
+        "positive": 22,
+        "negative": 29
+      },
+      "security alert": {
+        "positive": 2,
+        "negative": 68
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "login to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 71,
+        "negative": 7
+      },
+      "within 24 hours": {
+        "positive": 63,
+        "negative": 2
+      },
+      "within 48 hours": {
+        "positive": 4,
+        "negative": 3
+      }
+    },
+    "label_purity": 0.6106,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 155,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 243
+    },
+    "threeway_accuracy": null,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 0.9673366834170855,
+      "precision": 0.952755905511811,
+      "recall": 0.9958847736625515,
+      "f1": 0.9738430583501007,
+      "specificity": 0.9225806451612903,
+      "false_positive_rate": 0.07741935483870968,
+      "false_negative_rate": 0.00411522633744856,
+      "confusion_matrix": {
+        "tn": 143,
+        "fp": 12,
+        "fn": 1,
+        "tp": 242
+      },
+      "n": 398,
+      "n_positive": 243,
+      "n_negative": 155,
+      "roc_auc": 0.9959644232045666
+    }
+  },
+  "per_source": {
+    "TREC-06": {
+      "threshold": 0.5,
+      "accuracy": 0.9483498957183167,
+      "precision": 0.8401694159607669,
+      "recall": 0.9678993323061119,
+      "f1": 0.8995226730310263,
+      "specificity": 0.9422147001934236,
+      "false_positive_rate": 0.0577852998065764,
+      "false_negative_rate": 0.03210066769388803,
+      "confusion_matrix": {
+        "tn": 11691,
+        "fp": 717,
+        "fn": 125,
+        "tp": 3769
+      },
+      "n": 16302,
+      "n_positive": 3894,
+      "n_negative": 12408,
+      "roc_auc": 0.990803500202166,
+      "label3_counts": {
+        "legit": 12408,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 3894
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 3769,
+        "mean_tokens_full": 285.9,
+        "mean_tokens_fed": 232.6,
+        "median_tokens_full": 207.0,
+        "pct_truncated": 15.5
+      },
+      "tn": {
+        "n": 11691,
+        "mean_tokens_full": 853.9,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 267.0,
+        "pct_truncated": 23.5
+      },
+      "fp": {
+        "n": 717,
+        "mean_tokens_full": 685.8,
+        "mean_tokens_fed": 293.1,
+        "median_tokens_full": 311.0,
+        "pct_truncated": 31.7
+      },
+      "fn": {
+        "n": 125,
+        "mean_tokens_full": 255.8,
+        "mean_tokens_fed": 210.7,
+        "median_tokens_full": 187.0,
+        "pct_truncated": 8.0
+      },
+      "correct": {
+        "n": 15460,
+        "mean_tokens_full": 715.4,
+        "mean_tokens_fed": 278.8,
+        "median_tokens_full": 245.0,
+        "pct_truncated": 21.6
+      },
+      "wrong": {
+        "n": 842,
+        "mean_tokens_full": 622.0,
+        "mean_tokens_fed": 280.9,
+        "median_tokens_full": 256.0,
+        "pct_truncated": 28.1
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 1363,
+        "n_positive": 587,
+        "positive_rate": 0.43066764490095377,
+        "recall": 0.9829642248722317,
+        "fpr": 0.163659793814433,
+        "accuracy": 0.8994864269992663
+      },
+      {
+        "bucket": "65-128",
+        "n": 2538,
+        "n_positive": 736,
+        "positive_rate": 0.2899921197793538,
+        "recall": 0.9565217391304348,
+        "fpr": 0.050499445061043285,
+        "accuracy": 0.9515366430260047
+      },
+      {
+        "bucket": "129-256",
+        "n": 4594,
+        "n_positive": 1196,
+        "positive_rate": 0.260339573356552,
+        "recall": 0.9548494983277592,
+        "fpr": 0.03207769276044732,
+        "accuracy": 0.9645189377448846
+      },
+      {
+        "bucket": "257-512",
+        "n": 4232,
+        "n_positive": 780,
+        "positive_rate": 0.1843100189035917,
+        "recall": 0.9756410256410256,
+        "fpr": 0.047219003476245656,
+        "accuracy": 0.9569943289224953
+      },
+      {
+        "bucket": ">512",
+        "n": 3575,
+        "n_positive": 595,
+        "positive_rate": 0.16643356643356644,
+        "recall": 0.9831932773109243,
+        "fpr": 0.0761744966442953,
+        "accuracy": 0.9337062937062937
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 16302,
+    "seconds": 57.74,
+    "rows_per_s": 282.3,
+    "padded_tokens": 4606544,
+    "real_tokens": 4546874,
+    "padded_tokens_per_s": 79776,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 10.05,
+      "p50_ms": 8.9,
+      "p95_ms": 13.49,
+      "max_ms": 19.65,
+      "avg_s": 0.01,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.136,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 481.35,
+      "p50_ms": 464.28,
+      "p95_ms": 710.31,
+      "max_ms": 768.87,
+      "avg_s": 0.4813,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 6.505,
+      "threads": 16,
+      "estimated_full_run_s": 47.7
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-synthetic/best",
+    "run_name": "ft-synthetic",
+    "params": 337444867,
+    "timestamp": "2026-10-09 08:16",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-synthetic/error_analysis.md",
+    "n_fn": 125,
+    "n_fp": 717,
+    "composition": {
+      "fp": {
+        "n": 717,
+        "mailing_list_or_digest": 0.237,
+        "html_heavy": 0.001,
+        "very_short_lt20_tokens": 0.046,
+        "truncated_gt512": 0.317,
+        "lure_phrase": 0.017
+      },
+      "tn": {
+        "n": 11691,
+        "mailing_list_or_digest": 0.296,
+        "html_heavy": 0.002,
+        "very_short_lt20_tokens": 0.003,
+        "truncated_gt512": 0.235,
+        "lure_phrase": 0.012
+      },
+      "fn": {
+        "n": 125,
+        "mailing_list_or_digest": 0.072,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.08,
+        "lure_phrase": 0.008
+      },
+      "tp": {
+        "n": 3769,
+        "mailing_list_or_digest": 0.028,
+        "html_heavy": 0.001,
+        "very_short_lt20_tokens": 0.002,
+        "truncated_gt512": 0.155,
+        "lure_phrase": 0.064
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "blocki",
+        "share_a": 0.096,
+        "share_b": 0.0,
+        "ratio": 750.9
+      },
+      {
+        "token": "scanning",
+        "share_a": 0.096,
+        "share_b": 0.0,
+        "ratio": 750.9
+      },
+      {
+        "token": "progra",
+        "share_a": 0.096,
+        "share_b": 0.0,
+        "ratio": 750.9
+      },
+      {
+        "token": "scanmail",
+        "share_a": 0.096,
+        "share_b": 0.0,
+        "ratio": 750.9
+      },
+      {
+        "token": "smailex",
+        "share_a": 0.096,
+        "share_b": 0.0,
+        "ratio": 750.9
+      },
+      {
+        "token": "enroll",
+        "share_a": 0.072,
+        "share_b": 0.0,
+        "ratio": 570.68
+      },
+      {
+        "token": "non-active",
+        "share_a": 0.048,
+        "share_b": 0.0,
+        "ratio": 390.47
+      },
+      {
+        "token": "online-university",
+        "share_a": 0.048,
+        "share_b": 0.0,
+        "ratio": 390.47
+      },
+      {
+        "token": "matched",
+        "share_a": 0.096,
+        "share_b": 0.0,
+        "ratio": 250.3
+      },
+      {
+        "token": "tex-fonts",
+        "share_a": 0.064,
+        "share_b": 0.0,
+        "ratio": 170.2
+      },
+      {
+        "token": "blocking",
+        "share_a": 0.104,
+        "share_b": 0.001,
+        "ratio": 162.19
+      },
+      {
+        "token": "pattern",
+        "share_a": 0.104,
+        "share_b": 0.001,
+        "ratio": 162.19
+      },
+      {
+        "token": "enrolled",
+        "share_a": 0.048,
+        "share_b": 0.0,
+        "ratio": 130.16
+      },
+      {
+        "token": "tpn",
+        "share_a": 0.056,
+        "share_b": 0.001,
+        "ratio": 90.11
+      },
+      {
+        "token": "equivalent",
+        "share_a": 0.056,
+        "share_b": 0.001,
+        "ratio": 90.11
+      },
+      {
+        "token": "glance",
+        "share_a": 0.048,
+        "share_b": 0.001,
+        "ratio": 78.09
+      },
+      {
+        "token": "setting",
+        "share_a": 0.104,
+        "share_b": 0.001,
+        "ratio": 73.72
+      },
+      {
+        "token": "pif",
+        "share_a": 0.064,
+        "share_b": 0.001,
+        "ratio": 56.73
+      },
+      {
+        "token": "detected",
+        "share_a": 0.096,
+        "share_b": 0.002,
+        "ratio": 50.06
+      },
+      {
+        "token": "utah",
+        "share_a": 0.096,
+        "share_b": 0.002,
+        "ratio": 50.06
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "https",
+        "share_a": 0.106,
+        "share_b": 0.004,
+        "ratio": 26.24
+      },
+      {
+        "token": "editorial",
+        "share_a": 0.056,
+        "share_b": 0.003,
+        "ratio": 16.29
+      },
+      {
+        "token": "cleaned",
+        "share_a": 0.049,
+        "share_b": 0.005,
+        "ratio": 10.24
+      },
+      {
+        "token": "manage",
+        "share_a": 0.057,
+        "share_b": 0.006,
+        "ratio": 9.73
+      },
+      {
+        "token": "firewall",
+        "share_a": 0.049,
+        "share_b": 0.005,
+        "ratio": 9.11
+      },
+      {
+        "token": "password",
+        "share_a": 0.049,
+        "share_b": 0.006,
+        "ratio": 8.21
+      },
+      {
+        "token": "helo",
+        "share_a": 0.054,
+        "share_b": 0.007,
+        "ratio": 7.9
+      },
+      {
+        "token": "digest",
+        "share_a": 0.052,
+        "share_b": 0.007,
+        "ratio": 7.5
+      },
+      {
+        "token": "coordinator",
+        "share_a": 0.068,
+        "share_b": 0.01,
+        "ratio": 6.98
+      },
+      {
+        "token": "inform",
+        "share_a": 0.059,
+        "share_b": 0.009,
+        "ratio": 6.69
+      },
+      {
+        "token": "vision",
+        "share_a": 0.049,
+        "share_b": 0.007,
+        "ratio": 6.69
+      },
+      {
+        "token": "association",
+        "share_a": 0.06,
+        "share_b": 0.01,
+        "ratio": 5.88
+      },
+      {
+        "token": "contents",
+        "share_a": 0.067,
+        "share_b": 0.012,
+        "ratio": 5.71
+      },
+      {
+        "token": "esmtp",
+        "share_a": 0.081,
+        "share_b": 0.014,
+        "ratio": 5.66
+      },
+      {
+        "token": "precedence",
+        "share_a": 0.056,
+        "share_b": 0.01,
+        "ratio": 5.62
+      },
+      {
+        "token": "content-transfer-encoding",
+        "share_a": 0.078,
+        "share_b": 0.014,
+        "ratio": 5.43
+      },
+      {
+        "token": "administrative",
+        "share_a": 0.05,
+        "share_b": 0.009,
+        "ratio": 5.38
+      },
+      {
+        "token": "forwarding",
+        "share_a": 0.081,
+        "share_b": 0.015,
+        "ratio": 5.34
+      },
+      {
+        "token": "lists",
+        "share_a": 0.112,
+        "share_b": 0.022,
+        "ratio": 5.17
+      },
+      {
+        "token": "mime-version",
+        "share_a": 0.092,
+        "share_b": 0.018,
+        "ratio": 5.17
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:052869",
+      "primary:020947",
+      "primary:165463",
+      "primary:071577",
+      "primary:168340",
+      "primary:116583",
+      "primary:080991",
+      "primary:086987",
+      "primary:006858",
+      "primary:143065",
+      "primary:175365",
+      "primary:000305",
+      "primary:202896",
+      "primary:098454",
+      "primary:044401",
+      "primary:022080",
+      "primary:167537",
+      "primary:119078",
+      "primary:122099",
+      "primary:161198",
+      "primary:056767",
+      "primary:121268",
+      "primary:191646",
+      "primary:064709",
+      "primary:179180"
+    ],
+    "top_fp_doc_ids": [
+      "primary:052736",
+      "primary:143547",
+      "primary:202546",
+      "primary:174983",
+      "primary:159590",
+      "primary:084432",
+      "primary:048923",
+      "primary:170605",
+      "primary:136645",
+      "primary:104918",
+      "primary:094893",
+      "primary:111352",
+      "primary:070632",
+      "primary:167744",
+      "primary:003978",
+      "primary:129925",
+      "primary:159954",
+      "primary:041464",
+      "primary:040048",
+      "primary:011014",
+      "primary:027800",
+      "primary:033174",
+      "primary:124126",
+      "primary:187169",
+      "primary:144257"
+    ],
+    "threeway": null
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-synthetic/confusion_matrix.png",
+    "roc_curve": "reports/eval_ft-synthetic/roc_curve.png",
+    "pr_curve": "reports/eval_ft-synthetic/pr_curve.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-synthetic/metrics_test3.json b/presentation/reports/eval_ft-synthetic/metrics_test3.json
new file mode 100644
index 0000000000000000000000000000000000000000..8fb5afb03e0abf3eb6c6ed6101f8f2374845be01
--- /dev/null
+++ b/presentation/reports/eval_ft-synthetic/metrics_test3.json
@@ -0,0 +1,1730 @@
+{
+  "timestamp": "2026-10-09 08:16",
+  "run_name": "ft-synthetic",
+  "split": "test3",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout3.parquet",
+  "split_title": "three-way holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-synthetic/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-synthetic",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "cross_entropy",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "on",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 0,
+    "best_step": 5000,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9926514877725575,
+    "val_metrics": {
+      "accuracy": 0.9920177963883801,
+      "precision": 0.9903846153846154,
+      "recall": 0.9949287611687998,
+      "f1": 0.9926514877725575,
+      "roc_auc": 0.9996055651398734,
+      "pr_auc": 0.9996662445428656,
+      "tn": 6922,
+      "fp": 80,
+      "fn": 42,
+      "tp": 8240,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.6093423366546631,
+      "f1_at_optimal_threshold": 0.9930643507629214,
+      "f1_unwanted": 0.9926514877725575,
+      "n_3way": 8105,
+      "accuracy_3way": 0.9891425046267736,
+      "macro_f1_3way": 0.9665378162175431,
+      "precision_legitimate": 0.9992783951508154,
+      "recall_legitimate": 0.988860325621251,
+      "f1_legitimate": 0.9940420644605555,
+      "precision_phishing": 0.8805194805194805,
+      "recall_phishing": 0.9970588235294118,
+      "f1_phishing": 0.9351724137931035,
+      "precision_spam": 0.9532237673830595,
+      "recall_spam": 0.9882044560943644,
+      "f1_spam": 0.9703989703989704,
+      "cm3_ll": 6924,
+      "cm3_lp": 41,
+      "cm3_ls": 37,
+      "cm3_pl": 1,
+      "cm3_pp": 339,
+      "cm3_ps": 0,
+      "cm3_sl": 4,
+      "cm3_sp": 5,
+      "cm3_ss": 754,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 340,
+      "pvs_phish_as_spam": 0,
+      "pvs_spam_as_phish": 6,
+      "pvs_spam_as_spam": 757,
+      "val_loss": 0.027758434301769195,
+      "n": 15284,
+      "eval_seconds": 47.8
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9920177963883801,
+        "precision": 0.9903846153846154,
+        "recall": 0.9949287611687998,
+        "f1": 0.9926514877725575,
+        "roc_auc": 0.9996055651398734,
+        "pr_auc": 0.9996662445428656,
+        "tn": 6922,
+        "fp": 80,
+        "fn": 42,
+        "tp": 8240,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.6093423366546631,
+        "f1_at_optimal_threshold": 0.9930643507629214,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.9891425046267736,
+        "macro_f1": 0.9665378162175431,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9992783951508154,
+            "recall": 0.988860325621251,
+            "f1": 0.9940420644605555
+          },
+          "Phishing": {
+            "precision": 0.8805194805194805,
+            "recall": 0.9970588235294118,
+            "f1": 0.9351724137931035
+          },
+          "Spam": {
+            "precision": 0.9532237673830595,
+            "recall": 0.9882044560943644,
+            "f1": 0.9703989703989704
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6924,
+              41,
+              37
+            ],
+            [
+              1,
+              339,
+              0
+            ],
+            [
+              4,
+              5,
+              754
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              340,
+              0
+            ],
+            [
+              6,
+              757
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.6093423366546631,
+      "val_f1_at_optimal": 0.9930643507629214
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 3220,
+    "positive": 1221,
+    "negative": 1999,
+    "positive_rate": 0.3792,
+    "label3": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "by_source": {
+      "Assassin": {
+        "rows": 236,
+        "positive": 72,
+        "label3": {
+          "legit": 164,
+          "phish": 0,
+          "spam": 72,
+          "unwanted": 0
+        }
+      },
+      "CEAS-08": {
+        "rows": 736,
+        "positive": 0,
+        "label3": {
+          "legit": 736,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Enron": {
+        "rows": 628,
+        "positive": 628,
+        "label3": {
+          "legit": 0,
+          "phish": 0,
+          "spam": 628,
+          "unwanted": 0
+        }
+      },
+      "Nazario": {
+        "rows": 222,
+        "positive": 222,
+        "label3": {
+          "legit": 0,
+          "phish": 222,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Nigerian_Fraud": {
+        "rows": 299,
+        "positive": 299,
+        "label3": {
+          "legit": 0,
+          "phish": 299,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "TREC-07": {
+        "rows": 1099,
+        "positive": 0,
+        "label3": {
+          "legit": 1099,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      }
+    },
+    "mean_tokens_full": 835.2,
+    "pct_truncated": 34.01
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9903726708074534,
+    "precision": 0.9861111111111112,
+    "recall": 0.9885339885339886,
+    "f1": 0.9873210633946831,
+    "specificity": 0.991495747873937,
+    "false_positive_rate": 0.008504252126063032,
+    "false_negative_rate": 0.011466011466011465,
+    "confusion_matrix": {
+      "tn": 1982,
+      "fp": 17,
+      "fn": 14,
+      "tp": 1207
+    },
+    "n": 3220,
+    "n_positive": 1221,
+    "n_negative": 1999
+  },
+  "threshold_free": {
+    "roc_auc": 0.9992785090333864,
+    "pr_auc": 0.99880958438229,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.9885339885339886,
+      "fpr": 0.008504252126063032,
+      "threshold": 0.5084403157234192,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.8394758394758395,
+      "fpr": 0.0005002501250625312,
+      "threshold": 0.9995763897895813,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.5084403157234192,
+      "f1": 0.9873210633946831,
+      "precision": 0.9861111111111112,
+      "recall": 0.9885339885339886
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 1207,
+        "predicted_phishing": 525,
+        "predicted_spam": 682
+      },
+      "false_positives_flagged": {
+        "n": 17,
+        "predicted_phishing": 6,
+        "predicted_spam": 11
+      },
+      "missed_positives": {
+        "n": 14,
+        "predicted_phishing": 2,
+        "predicted_spam": 12
+      },
+      "all_flagged": {
+        "n": 1224,
+        "predicted_phishing": 531,
+        "predicted_spam": 693
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 0,
+        "predicted_phishing": 0,
+        "predicted_spam": 0
+      }
+    },
+    "rows_with_full_label": 3220,
+    "rows_unwanted_only": 0,
+    "evaluated": true,
+    "reason": null,
+    "confusion_matrix": {
+      "labels": [
+        "Legitimate",
+        "Phishing",
+        "Spam"
+      ],
+      "matrix": [
+        [
+          1982,
+          6,
+          11
+        ],
+        [
+          4,
+          514,
+          3
+        ],
+        [
+          10,
+          11,
+          679
+        ]
+      ],
+      "cells": {
+        "true_legitimate": {
+          "pred_legitimate": 1982,
+          "pred_phishing": 6,
+          "pred_spam": 11
+        },
+        "true_phishing": {
+          "pred_legitimate": 4,
+          "pred_phishing": 514,
+          "pred_spam": 3
+        },
+        "true_spam": {
+          "pred_legitimate": 10,
+          "pred_phishing": 11,
+          "pred_spam": 679
+        }
+      }
+    },
+    "accuracy": 0.9860248447204969,
+    "macro_f1": 0.9814336613406126,
+    "macro_f1_present_classes": 0.9814336613406126,
+    "classes_present": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "per_class": {
+      "Legitimate": {
+        "precision": 0.9929859719438878,
+        "recall": 0.991495747873937,
+        "f1": 0.9922403003754693,
+        "support": 1999
+      },
+      "Phishing": {
+        "precision": 0.967984934086629,
+        "recall": 0.9865642994241842,
+        "f1": 0.9771863117870723,
+        "support": 521
+      },
+      "Spam": {
+        "precision": 0.9797979797979798,
+        "recall": 0.97,
+        "f1": 0.9748743718592965,
+        "support": 700
+      }
+    },
+    "argmax_rule": {
+      "doc": "plain argmax over the three probabilities (secondary; can disagree with the binary verdict near the threshold)",
+      "accuracy": 0.9860248447204969,
+      "macro_f1": 0.9814336613406126,
+      "per_class": {
+        "Legitimate": {
+          "precision": 0.9929859719438878,
+          "recall": 0.991495747873937,
+          "f1": 0.9922403003754693,
+          "support": 1999
+        },
+        "Phishing": {
+          "precision": 0.967984934086629,
+          "recall": 0.9865642994241842,
+          "f1": 0.9771863117870723,
+          "support": 521
+        },
+        "Spam": {
+          "precision": 0.9797979797979798,
+          "recall": 0.97,
+          "f1": 0.9748743718592965,
+          "support": 700
+        }
+      },
+      "rows_where_rules_differ": 0
+    },
+    "phish_vs_spam_on_true_unwanted": {
+      "n_true_unwanted": 1221,
+      "kind_only": {
+        "doc": "given the row is unwanted, is argmax(P(phishing), P(spam)) right? (independent of the roll-up threshold)",
+        "labels": [
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            516,
+            5
+          ],
+          [
+            11,
+            689
+          ]
+        ],
+        "cells": {
+          "true_phishing": {
+            "pred_phishing": 516,
+            "pred_spam": 5
+          },
+          "true_spam": {
+            "pred_phishing": 11,
+            "pred_spam": 689
+          }
+        },
+        "accuracy": 0.9868959868959869,
+        "phishing_as_spam": 5,
+        "spam_as_phishing": 11
+      },
+      "rollup_rule": {
+        "doc": "rows = true Phishing / Spam; columns = predicted Legitimate (missed) / Phishing / Spam under the roll-up rule",
+        "rows": [
+          "Phishing",
+          "Spam"
+        ],
+        "columns": [
+          "Legitimate",
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            4,
+            514,
+            3
+          ],
+          [
+            10,
+            11,
+            679
+          ]
+        ],
+        "missed_as_legitimate": {
+          "phishing": 4,
+          "spam": 10
+        }
+      }
+    },
+    "per_source": {
+      "Assassin": {
+        "n": 236,
+        "counts": {
+          "Legitimate": 164,
+          "Phishing": 0,
+          "Spam": 72
+        },
+        "accuracy": 0.9279661016949152,
+        "macro_f1": 0.6197300956988756,
+        "macro_f1_present_classes": 0.9295951435483134,
+        "per_class_recall": {
+          "Legitimate": 0.975609756097561,
+          "Spam": 0.8194444444444444
+        }
+      },
+      "CEAS-08": {
+        "n": 736,
+        "counts": {
+          "Legitimate": 736,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9850543478260869,
+        "macro_f1": 0.3308236367784622,
+        "macro_f1_present_classes": 0.9924709103353867,
+        "per_class_recall": {
+          "Legitimate": 0.9850543478260869
+        }
+      },
+      "Enron": {
+        "n": 628,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 0,
+          "Spam": 628
+        },
+        "accuracy": 0.9872611464968153,
+        "macro_f1": 0.3311965811965812,
+        "macro_f1_present_classes": 0.9935897435897436,
+        "per_class_recall": {
+          "Spam": 0.9872611464968153
+        }
+      },
+      "Nazario": {
+        "n": 222,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 222,
+          "Spam": 0
+        },
+        "accuracy": 0.9819819819819819,
+        "macro_f1": 0.3303030303030303,
+        "macro_f1_present_classes": 0.990909090909091,
+        "per_class_recall": {
+          "Phishing": 0.9819819819819819
+        }
+      },
+      "Nigerian_Fraud": {
+        "n": 299,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 299,
+          "Spam": 0
+        },
+        "accuracy": 0.9899665551839465,
+        "macro_f1": 0.33165266106442576,
+        "macro_f1_present_classes": 0.9949579831932773,
+        "per_class_recall": {
+          "Phishing": 0.9899665551839465
+        }
+      },
+      "TREC-07": {
+        "n": 1099,
+        "counts": {
+          "Legitimate": 1099,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9981801637852593,
+        "macro_f1": 0.33302975106253796,
+        "macro_f1_present_classes": 0.9990892531876139,
+        "per_class_recall": {
+          "Legitimate": 0.9981801637852593
+        }
+      }
+    }
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 139,
+      "bank account": 91,
+      "urgently": 25,
+      "verify your account": 15,
+      "confirm your account": 9,
+      "update your account": 9,
+      "within 24 hours": 8,
+      "security alert": 8,
+      "paypal account": 7,
+      "password will expire": 7,
+      "click here to verify": 6,
+      "unusual activity": 4,
+      "wire transfer": 3,
+      "update your information": 3,
+      "gift card": 3,
+      "verify your identity": 2,
+      "unusual sign-in": 2,
+      "account will be suspended": 2,
+      "update your payment": 2,
+      "failure to verify": 2,
+      "within 48 hours": 2,
+      "confirm your identity": 2,
+      "enter your password": 1,
+      "update your billing": 1,
+      "reset your password": 1,
+      "suspicious activity": 1,
+      "security notice": 1,
+      "log in to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 15,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "update your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 2,
+        "negative": 1
+      },
+      "update your billing": {
+        "positive": 1,
+        "negative": 0
+      },
+      "update your payment": {
+        "positive": 2,
+        "negative": 0
+      },
+      "click here to verify": {
+        "positive": 6,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual sign-in": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "suspicious activity": {
+        "positive": 1,
+        "negative": 0
+      },
+      "paypal account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "password will expire": {
+        "positive": 7,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 1,
+        "negative": 0
+      },
+      "wire transfer": {
+        "positive": 2,
+        "negative": 1
+      },
+      "gift card": {
+        "positive": 3,
+        "negative": 0
+      },
+      "urgent": {
+        "positive": 133,
+        "negative": 6
+      },
+      "urgently": {
+        "positive": 25,
+        "negative": 0
+      },
+      "security alert": {
+        "positive": 8,
+        "negative": 0
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 91,
+        "negative": 0
+      },
+      "failure to verify": {
+        "positive": 2,
+        "negative": 0
+      },
+      "within 24 hours": {
+        "positive": 8,
+        "negative": 0
+      },
+      "within 48 hours": {
+        "positive": 2,
+        "negative": 0
+      }
+    },
+    "label_purity": 0.9745,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 7,
+      "phish": 236,
+      "spam": 32,
+      "unwanted": 0
+    },
+    "threeway_accuracy": 0.9781818181818182,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 1.0,
+      "precision": 1.0,
+      "recall": 1.0,
+      "f1": 1.0,
+      "specificity": 1.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 7,
+        "fp": 0,
+        "fn": 0,
+        "tp": 268
+      },
+      "n": 275,
+      "n_positive": 268,
+      "n_negative": 7,
+      "roc_auc": 1.0
+    }
+  },
+  "per_source": {
+    "Assassin": {
+      "threshold": 0.5,
+      "accuracy": 0.9703389830508474,
+      "precision": 0.9452054794520548,
+      "recall": 0.9583333333333334,
+      "f1": 0.9517241379310345,
+      "specificity": 0.975609756097561,
+      "false_positive_rate": 0.024390243902439025,
+      "false_negative_rate": 0.041666666666666664,
+      "confusion_matrix": {
+        "tn": 160,
+        "fp": 4,
+        "fn": 3,
+        "tp": 69
+      },
+      "n": 236,
+      "n_positive": 72,
+      "n_negative": 164,
+      "roc_auc": 0.9952574525745257,
+      "label3_counts": {
+        "legit": 164,
+        "phish": 0,
+        "spam": 72,
+        "unwanted": 0
+      }
+    },
+    "CEAS-08": {
+      "threshold": 0.5,
+      "accuracy": 0.9850543478260869,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9850543478260869,
+      "false_positive_rate": 0.014945652173913044,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 725,
+        "fp": 11,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 736,
+      "n_positive": 0,
+      "n_negative": 736,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 736,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Enron": {
+      "threshold": 0.5,
+      "accuracy": 0.9888535031847133,
+      "precision": 1.0,
+      "recall": 0.9888535031847133,
+      "f1": 0.9943955164131305,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.011146496815286623,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 7,
+        "tp": 621
+      },
+      "n": 628,
+      "n_positive": 628,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 0,
+        "spam": 628,
+        "unwanted": 0
+      }
+    },
+    "Nazario": {
+      "threshold": 0.5,
+      "accuracy": 0.9864864864864865,
+      "precision": 1.0,
+      "recall": 0.9864864864864865,
+      "f1": 0.9931972789115647,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.013513513513513514,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 3,
+        "tp": 219
+      },
+      "n": 222,
+      "n_positive": 222,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 222,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Nigerian_Fraud": {
+      "threshold": 0.5,
+      "accuracy": 0.9966555183946488,
+      "precision": 1.0,
+      "recall": 0.9966555183946488,
+      "f1": 0.998324958123953,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.0033444816053511705,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 1,
+        "tp": 298
+      },
+      "n": 299,
+      "n_positive": 299,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 299,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "TREC-07": {
+      "threshold": 0.5,
+      "accuracy": 0.9981801637852593,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9981801637852593,
+      "false_positive_rate": 0.0018198362147406734,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 1097,
+        "fp": 2,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 1099,
+      "n_positive": 0,
+      "n_negative": 1099,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 1099,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 1207,
+        "mean_tokens_full": 998.0,
+        "mean_tokens_fed": 279.9,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.6
+      },
+      "tn": {
+        "n": 1982,
+        "mean_tokens_full": 734.1,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.3
+      },
+      "fp": {
+        "n": 17,
+        "mean_tokens_full": 1389.4,
+        "mean_tokens_fed": 347.8,
+        "median_tokens_full": 809.0,
+        "pct_truncated": 52.9
+      },
+      "fn": {
+        "n": 14,
+        "mean_tokens_full": 447.1,
+        "mean_tokens_fed": 280.3,
+        "median_tokens_full": 227.5,
+        "pct_truncated": 35.7
+      },
+      "correct": {
+        "n": 3189,
+        "mean_tokens_full": 834.0,
+        "mean_tokens_fed": 328.8,
+        "median_tokens_full": 339.0,
+        "pct_truncated": 33.9
+      },
+      "wrong": {
+        "n": 31,
+        "mean_tokens_full": 963.8,
+        "mean_tokens_fed": 317.3,
+        "median_tokens_full": 352.0,
+        "pct_truncated": 45.2
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "unwanted": {
+        "n": 1221,
+        "mean_tokens_full": 991.6,
+        "mean_tokens_fed": 279.9,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.7
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "phish": {
+        "n": 521,
+        "mean_tokens_full": 1349.4,
+        "mean_tokens_fed": 355.1,
+        "median_tokens_full": 412.0,
+        "pct_truncated": 39.9
+      },
+      "spam": {
+        "n": 700,
+        "mean_tokens_full": 725.3,
+        "mean_tokens_fed": 223.9,
+        "median_tokens_full": 159.0,
+        "pct_truncated": 16.9
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 129,
+        "n_positive": 99,
+        "positive_rate": 0.7674418604651163,
+        "recall": 0.9797979797979798,
+        "fpr": 0.06666666666666667,
+        "accuracy": 0.9689922480620154
+      },
+      {
+        "bucket": "65-128",
+        "n": 427,
+        "n_positive": 252,
+        "positive_rate": 0.5901639344262295,
+        "recall": 0.9920634920634921,
+        "fpr": 0.011428571428571429,
+        "accuracy": 0.990632318501171
+      },
+      {
+        "bucket": "129-256",
+        "n": 699,
+        "n_positive": 276,
+        "positive_rate": 0.3948497854077253,
+        "recall": 0.9891304347826086,
+        "fpr": 0.004728132387706856,
+        "accuracy": 0.9928469241773963
+      },
+      {
+        "bucket": "257-512",
+        "n": 870,
+        "n_positive": 268,
+        "positive_rate": 0.3080459770114943,
+        "recall": 0.9925373134328358,
+        "fpr": 0.0033222591362126247,
+        "accuracy": 0.9954022988505747
+      },
+      {
+        "bucket": ">512",
+        "n": 1095,
+        "n_positive": 326,
+        "positive_rate": 0.29771689497716897,
+        "recall": 0.9846625766871165,
+        "fpr": 0.011703511053315995,
+        "accuracy": 0.9872146118721461
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 3220,
+    "seconds": 13.51,
+    "rows_per_s": 238.3,
+    "padded_tokens": 1079936,
+    "real_tokens": 1058490,
+    "padded_tokens_per_s": 79934,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 10.05,
+      "p50_ms": 8.9,
+      "p95_ms": 13.49,
+      "max_ms": 19.65,
+      "avg_s": 0.01,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.136,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 481.35,
+      "p50_ms": 464.28,
+      "p95_ms": 710.31,
+      "max_ms": 768.87,
+      "avg_s": 0.4813,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 6.505,
+      "threads": 16,
+      "estimated_full_run_s": 47.7
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-synthetic/best",
+    "run_name": "ft-synthetic",
+    "params": 337444867,
+    "timestamp": "2026-10-09 08:16",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-synthetic/error_analysis_test3.md",
+    "n_fn": 14,
+    "n_fp": 17,
+    "composition": {
+      "fp": {
+        "n": 17,
+        "mailing_list_or_digest": 0.412,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.529,
+        "lure_phrase": 0.0
+      },
+      "tn": {
+        "n": 1982,
+        "mailing_list_or_digest": 0.564,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.383,
+        "lure_phrase": 0.004
+      },
+      "fn": {
+        "n": 14,
+        "mailing_list_or_digest": 0.286,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.357,
+        "lure_phrase": 0.0
+      },
+      "tp": {
+        "n": 1207,
+        "mailing_list_or_digest": 0.085,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.007,
+        "truncated_gt512": 0.266,
+        "lure_phrase": 0.222
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "invite",
+        "share_a": 0.286,
+        "share_b": 0.002,
+        "ratio": 107.07
+      },
+      {
+        "token": "angeles",
+        "share_a": 0.214,
+        "share_b": 0.004,
+        "ratio": 52.99
+      },
+      {
+        "token": "york",
+        "share_a": 0.214,
+        "share_b": 0.005,
+        "ratio": 44.84
+      },
+      {
+        "token": "event",
+        "share_a": 0.214,
+        "share_b": 0.006,
+        "ratio": 38.86
+      },
+      {
+        "token": "technologies",
+        "share_a": 0.214,
+        "share_b": 0.006,
+        "ratio": 38.86
+      },
+      {
+        "token": "entertainment",
+        "share_a": 0.214,
+        "share_b": 0.007,
+        "ratio": 34.29
+      },
+      {
+        "token": "association",
+        "share_a": 0.214,
+        "share_b": 0.008,
+        "ratio": 27.76
+      },
+      {
+        "token": "subscription",
+        "share_a": 0.214,
+        "share_b": 0.008,
+        "ratio": 27.76
+      },
+      {
+        "token": "sound",
+        "share_a": 0.214,
+        "share_b": 0.011,
+        "ratio": 21.59
+      },
+      {
+        "token": "staff",
+        "share_a": 0.214,
+        "share_b": 0.012,
+        "ratio": 20.1
+      },
+      {
+        "token": "changes",
+        "share_a": 0.214,
+        "share_b": 0.012,
+        "ratio": 18.8
+      },
+      {
+        "token": "cards",
+        "share_a": 0.214,
+        "share_b": 0.015,
+        "ratio": 15.75
+      },
+      {
+        "token": "media",
+        "share_a": 0.214,
+        "share_b": 0.015,
+        "ratio": 15.75
+      },
+      {
+        "token": "leading",
+        "share_a": 0.214,
+        "share_b": 0.015,
+        "ratio": 15.75
+      },
+      {
+        "token": "los",
+        "share_a": 0.214,
+        "share_b": 0.016,
+        "ratio": 14.95
+      },
+      {
+        "token": "join",
+        "share_a": 0.286,
+        "share_b": 0.021,
+        "ratio": 14.7
+      },
+      {
+        "token": "friends",
+        "share_a": 0.214,
+        "share_b": 0.022,
+        "ratio": 11.0
+      },
+      {
+        "token": "january",
+        "share_a": 0.214,
+        "share_b": 0.023,
+        "ratio": 10.23
+      },
+      {
+        "token": "simply",
+        "share_a": 0.214,
+        "share_b": 0.026,
+        "ratio": 9.25
+      },
+      {
+        "token": "stock",
+        "share_a": 0.214,
+        "share_b": 0.026,
+        "ratio": 9.25
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "cook",
+        "share_a": 0.176,
+        "share_b": 0.001,
+        "ratio": 158.6
+      },
+      {
+        "token": "full-color",
+        "share_a": 0.235,
+        "share_b": 0.003,
+        "ratio": 92.69
+      },
+      {
+        "token": "retail",
+        "share_a": 0.176,
+        "share_b": 0.002,
+        "ratio": 88.11
+      },
+      {
+        "token": "pie",
+        "share_a": 0.176,
+        "share_b": 0.003,
+        "ratio": 72.09
+      },
+      {
+        "token": "obligation",
+        "share_a": 0.176,
+        "share_b": 0.004,
+        "ratio": 52.87
+      },
+      {
+        "token": "subscriber",
+        "share_a": 0.176,
+        "share_b": 0.004,
+        "ratio": 46.65
+      },
+      {
+        "token": "gift",
+        "share_a": 0.176,
+        "share_b": 0.005,
+        "ratio": 41.74
+      },
+      {
+        "token": "serves",
+        "share_a": 0.176,
+        "share_b": 0.005,
+        "ratio": 37.76
+      },
+      {
+        "token": "fraud",
+        "share_a": 0.176,
+        "share_b": 0.006,
+        "ratio": 34.48
+      },
+      {
+        "token": "multi-part",
+        "share_a": 0.176,
+        "share_b": 0.006,
+        "ratio": 31.72
+      },
+      {
+        "token": "content-transfer-encoding",
+        "share_a": 0.176,
+        "share_b": 0.008,
+        "ratio": 25.58
+      },
+      {
+        "token": "identity",
+        "share_a": 0.176,
+        "share_b": 0.008,
+        "ratio": 25.58
+      },
+      {
+        "token": "opportunities",
+        "share_a": 0.176,
+        "share_b": 0.008,
+        "ratio": 24.03
+      },
+      {
+        "token": "prior",
+        "share_a": 0.235,
+        "share_b": 0.011,
+        "ratio": 22.66
+      },
+      {
+        "token": "located",
+        "share_a": 0.176,
+        "share_b": 0.009,
+        "ratio": 22.66
+      },
+      {
+        "token": "advertising",
+        "share_a": 0.235,
+        "share_b": 0.012,
+        "ratio": 21.69
+      },
+      {
+        "token": "minute",
+        "share_a": 0.176,
+        "share_b": 0.009,
+        "ratio": 21.43
+      },
+      {
+        "token": "executive",
+        "share_a": 0.176,
+        "share_b": 0.009,
+        "ratio": 21.43
+      },
+      {
+        "token": "win",
+        "share_a": 0.235,
+        "share_b": 0.012,
+        "ratio": 20.81
+      },
+      {
+        "token": "employees",
+        "share_a": 0.176,
+        "share_b": 0.01,
+        "ratio": 20.33
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:148887",
+      "aura:Nazario:000643",
+      "aura:Nazario:000097",
+      "primary:181246",
+      "primary:191966",
+      "aura:Nazario:000300",
+      "primary:007176",
+      "primary:102650",
+      "primary:001939",
+      "primary:010013",
+      "primary:046569",
+      "primary:095405",
+      "primary:026089",
+      "aura:Nigerian_Fraud:003049"
+    ],
+    "top_fp_doc_ids": [
+      "primary:091801",
+      "primary:144048",
+      "primary:048756",
+      "primary:116110",
+      "primary:152763",
+      "primary:199386",
+      "primary:041713",
+      "primary:027029",
+      "primary:146259",
+      "primary:021911",
+      "primary:165228",
+      "primary:103927",
+      "primary:153841",
+      "primary:096865",
+      "primary:057514",
+      "primary:156985",
+      "primary:055965"
+    ],
+    "threeway": {
+      "n_phish_as_spam": 3,
+      "n_spam_as_phish": 11,
+      "top_phish_as_spam_doc_ids": [
+        "aura:Nigerian_Fraud:001698",
+        "aura:Nigerian_Fraud:000337",
+        "aura:Nazario:000532"
+      ],
+      "top_spam_as_phish_doc_ids": [
+        "primary:077835",
+        "primary:197304",
+        "primary:201118",
+        "primary:168239",
+        "primary:028382",
+        "primary:169192",
+        "primary:153313",
+        "primary:198712",
+        "primary:120914",
+        "primary:129677",
+        "primary:136790"
+      ],
+      "phish_as_spam_tokens": [
+        {
+          "token": "complete",
+          "share_a": 1.0,
+          "share_b": 0.049,
+          "ratio": 20.18
+        },
+        {
+          "token": "www",
+          "share_a": 1.0,
+          "share_b": 0.15,
+          "ratio": 6.64
+        },
+        {
+          "token": "http",
+          "share_a": 1.0,
+          "share_b": 0.193,
+          "ratio": 5.17
+        },
+        {
+          "token": "com",
+          "share_a": 1.0,
+          "share_b": 0.337,
+          "ratio": 2.97
+        },
+        {
+          "token": "here",
+          "share_a": 1.0,
+          "share_b": 0.411,
+          "ratio": 2.43
+        },
+        {
+          "token": "are",
+          "share_a": 1.0,
+          "share_b": 0.564,
+          "ratio": 1.77
+        },
+        {
+          "token": "will",
+          "share_a": 1.0,
+          "share_b": 0.714,
+          "ratio": 1.4
+        },
+        {
+          "token": "you",
+          "share_a": 1.0,
+          "share_b": 0.891,
+          "ratio": 1.12
+        },
+        {
+          "token": "your",
+          "share_a": 1.0,
+          "share_b": 0.92,
+          "ratio": 1.09
+        }
+      ],
+      "spam_as_phish_tokens": [
+        {
+          "token": "mailman",
+          "share_a": 0.273,
+          "share_b": 0.013,
+          "ratio": 21.77
+        },
+        {
+          "token": "listinfo",
+          "share_a": 0.273,
+          "share_b": 0.018,
+          "ratio": 16.54
+        },
+        {
+          "token": "upon",
+          "share_a": 0.273,
+          "share_b": 0.031,
+          "ratio": 9.62
+        },
+        {
+          "token": "private",
+          "share_a": 0.273,
+          "share_b": 0.043,
+          "ratio": 7.01
+        },
+        {
+          "token": "help",
+          "share_a": 0.364,
+          "share_b": 0.062,
+          "ratio": 6.26
+        },
+        {
+          "token": "immediately",
+          "share_a": 0.273,
+          "share_b": 0.049,
+          "ratio": 6.17
+        },
+        {
+          "token": "account",
+          "share_a": 0.273,
+          "share_b": 0.05,
+          "ratio": 5.99
+        },
+        {
+          "token": "got",
+          "share_a": 0.273,
+          "share_b": 0.053,
+          "ratio": 5.67
+        },
+        {
+          "token": "contact",
+          "share_a": 0.364,
+          "share_b": 0.085,
+          "ratio": 4.55
+        },
+        {
+          "token": "last",
+          "share_a": 0.273,
+          "share_b": 0.068,
+          "ratio": 4.45
+        },
+        {
+          "token": "people",
+          "share_a": 0.273,
+          "share_b": 0.068,
+          "ratio": 4.45
+        },
+        {
+          "token": "company",
+          "share_a": 0.273,
+          "share_b": 0.069,
+          "ratio": 4.35
+        },
+        {
+          "token": "their",
+          "share_a": 0.364,
+          "share_b": 0.106,
+          "ratio": 3.67
+        },
+        {
+          "token": "know",
+          "share_a": 0.273,
+          "share_b": 0.091,
+          "ratio": 3.31
+        },
+        {
+          "token": "was",
+          "share_a": 0.364,
+          "share_b": 0.13,
+          "ratio": 3.0
+        },
+        {
+          "token": "net",
+          "share_a": 0.364,
+          "share_b": 0.133,
+          "ratio": 2.94
+        },
+        {
+          "token": "take",
+          "share_a": 0.273,
+          "share_b": 0.103,
+          "ratio": 2.93
+        },
+        {
+          "token": "they",
+          "share_a": 0.273,
+          "share_b": 0.103,
+          "ratio": 2.93
+        },
+        {
+          "token": "what",
+          "share_a": 0.273,
+          "share_b": 0.127,
+          "ratio": 2.39
+        },
+        {
+          "token": "only",
+          "share_a": 0.364,
+          "share_b": 0.165,
+          "ratio": 2.36
+        }
+      ]
+    }
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-synthetic/confusion_matrix_test3.png",
+    "confusion_matrix3": "reports/eval_ft-synthetic/confusion_matrix3_test3.png",
+    "roc_curve": "reports/eval_ft-synthetic/roc_curve_test3.png",
+    "pr_curve": "reports/eval_ft-synthetic/pr_curve_test3.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-weighted/metrics.json b/presentation/reports/eval_ft-weighted/metrics.json
new file mode 100644
index 0000000000000000000000000000000000000000..e7b344d9338601c4fff55e7f7600f6df611e82af
--- /dev/null
+++ b/presentation/reports/eval_ft-weighted/metrics.json
@@ -0,0 +1,1109 @@
+{
+  "timestamp": "2026-10-09 05:55",
+  "run_name": "ft-weighted",
+  "split": "test",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet",
+  "split_title": "TREC-06 binary holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-weighted/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-weighted",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "weighted_cross_entropy",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 1,
+    "best_step": 10000,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.99373795761079,
+    "val_metrics": {
+      "accuracy": 0.9931954985605862,
+      "precision": 0.9911121787172712,
+      "recall": 0.9963776865491427,
+      "f1": 0.99373795761079,
+      "roc_auc": 0.9997532357160727,
+      "pr_auc": 0.9997920087331971,
+      "tn": 6928,
+      "fp": 74,
+      "fn": 30,
+      "tp": 8252,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.8991281390190125,
+      "f1_at_optimal_threshold": 0.9942622455758894,
+      "f1_unwanted": 0.99373795761079,
+      "n_3way": 8105,
+      "accuracy_3way": 0.9902529302899444,
+      "macro_f1_3way": 0.9774841428658321,
+      "precision_legitimate": 0.9992789154888952,
+      "recall_legitimate": 0.9895744073121965,
+      "f1_legitimate": 0.9944029850746269,
+      "precision_phishing": 0.9630681818181818,
+      "recall_phishing": 0.9970588235294118,
+      "f1_phishing": 0.9797687861271677,
+      "precision_spam": 0.9255189255189256,
+      "recall_spam": 0.9934469200524246,
+      "f1_spam": 0.9582806573957017,
+      "cm3_ll": 6929,
+      "cm3_lp": 12,
+      "cm3_ls": 61,
+      "cm3_pl": 1,
+      "cm3_pp": 339,
+      "cm3_ps": 0,
+      "cm3_sl": 4,
+      "cm3_sp": 1,
+      "cm3_ss": 758,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 339,
+      "pvs_phish_as_spam": 1,
+      "pvs_spam_as_phish": 2,
+      "pvs_spam_as_spam": 761,
+      "val_loss": 0.025379083498832725,
+      "n": 15284,
+      "eval_seconds": 47.8
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9931954985605862,
+        "precision": 0.9911121787172712,
+        "recall": 0.9963776865491427,
+        "f1": 0.99373795761079,
+        "roc_auc": 0.9997532357160727,
+        "pr_auc": 0.9997920087331971,
+        "tn": 6928,
+        "fp": 74,
+        "fn": 30,
+        "tp": 8252,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.8991281390190125,
+        "f1_at_optimal_threshold": 0.9942622455758894,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.9902529302899444,
+        "macro_f1": 0.9774841428658321,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9992789154888952,
+            "recall": 0.9895744073121965,
+            "f1": 0.9944029850746269
+          },
+          "Phishing": {
+            "precision": 0.9630681818181818,
+            "recall": 0.9970588235294118,
+            "f1": 0.9797687861271677
+          },
+          "Spam": {
+            "precision": 0.9255189255189256,
+            "recall": 0.9934469200524246,
+            "f1": 0.9582806573957017
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6929,
+              12,
+              61
+            ],
+            [
+              1,
+              339,
+              0
+            ],
+            [
+              4,
+              1,
+              758
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              339,
+              1
+            ],
+            [
+              2,
+              761
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.8991281390190125,
+      "val_f1_at_optimal": 0.9942622455758894
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 16302,
+    "positive": 3894,
+    "negative": 12408,
+    "positive_rate": 0.2389,
+    "label3": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "by_source": {
+      "TREC-06": {
+        "rows": 16302,
+        "positive": 3894,
+        "label3": {
+          "legit": 12408,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 3894
+        }
+      }
+    },
+    "mean_tokens_full": 710.6,
+    "pct_truncated": 21.93
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9478591583854742,
+    "precision": 0.8307257714037375,
+    "recall": 0.9817668207498716,
+    "f1": 0.8999529190207156,
+    "specificity": 0.9372179239200515,
+    "false_positive_rate": 0.06278207607994842,
+    "false_negative_rate": 0.018233179250128403,
+    "confusion_matrix": {
+      "tn": 11629,
+      "fp": 779,
+      "fn": 71,
+      "tp": 3823
+    },
+    "n": 16302,
+    "n_positive": 3894,
+    "n_negative": 12408
+  },
+  "threshold_free": {
+    "roc_auc": 0.9942507931824557,
+    "pr_auc": 0.9832765259772845,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.8767334360554699,
+      "fpr": 0.009912959381044487,
+      "threshold": 0.9987348318099976,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.6335387776065742,
+      "fpr": 0.0009671179883945841,
+      "threshold": 0.9999274015426636,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.9889976382255554,
+      "f1": 0.938343949044586,
+      "precision": 0.9309908998988877,
+      "recall": 0.945814072932717
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 3823,
+        "predicted_phishing": 801,
+        "predicted_spam": 3022
+      },
+      "false_positives_flagged": {
+        "n": 779,
+        "predicted_phishing": 302,
+        "predicted_spam": 477
+      },
+      "missed_positives": {
+        "n": 71,
+        "predicted_phishing": 17,
+        "predicted_spam": 54
+      },
+      "all_flagged": {
+        "n": 4602,
+        "predicted_phishing": 1103,
+        "predicted_spam": 3499
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 3823,
+        "predicted_phishing": 801,
+        "predicted_spam": 3022
+      }
+    },
+    "rows_with_full_label": 12408,
+    "rows_unwanted_only": 3894,
+    "evaluated": false,
+    "reason": "no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)"
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 129,
+      "bank account": 78,
+      "security alert": 70,
+      "within 24 hours": 65,
+      "urgently": 51,
+      "paypal account": 20,
+      "within 48 hours": 7,
+      "verify your account": 7,
+      "update your account": 6,
+      "account will be suspended": 5,
+      "ebay account": 5,
+      "enter your password": 5,
+      "wire transfer": 5,
+      "unusual activity": 5,
+      "confirm your identity": 4,
+      "verify your identity": 4,
+      "click here to verify": 3,
+      "log in to your account": 2,
+      "confirm your account": 2,
+      "security notice": 1,
+      "gift card": 1,
+      "reset your password": 1,
+      "account has been locked": 1,
+      "update your information": 1,
+      "account has been limited": 1,
+      "login to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 5,
+        "negative": 0
+      },
+      "update your account": {
+        "positive": 6,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 0,
+        "negative": 1
+      },
+      "click here to verify": {
+        "positive": 3,
+        "negative": 0
+      },
+      "account has been limited": {
+        "positive": 1,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 4,
+        "negative": 1
+      },
+      "account has been locked": {
+        "positive": 1,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 2,
+        "negative": 3
+      },
+      "paypal account": {
+        "positive": 20,
+        "negative": 0
+      },
+      "ebay account": {
+        "positive": 5,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "wire transfer": {
+        "positive": 5,
+        "negative": 0
+      },
+      "gift card": {
+        "positive": 0,
+        "negative": 1
+      },
+      "urgent": {
+        "positive": 81,
+        "negative": 48
+      },
+      "urgently": {
+        "positive": 22,
+        "negative": 29
+      },
+      "security alert": {
+        "positive": 2,
+        "negative": 68
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "login to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 71,
+        "negative": 7
+      },
+      "within 24 hours": {
+        "positive": 63,
+        "negative": 2
+      },
+      "within 48 hours": {
+        "positive": 4,
+        "negative": 3
+      }
+    },
+    "label_purity": 0.6106,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 155,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 243
+    },
+    "threeway_accuracy": null,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 0.9698492462311558,
+      "precision": 0.9565217391304348,
+      "recall": 0.9958847736625515,
+      "f1": 0.9758064516129034,
+      "specificity": 0.9290322580645162,
+      "false_positive_rate": 0.07096774193548387,
+      "false_negative_rate": 0.00411522633744856,
+      "confusion_matrix": {
+        "tn": 144,
+        "fp": 11,
+        "fn": 1,
+        "tp": 242
+      },
+      "n": 398,
+      "n_positive": 243,
+      "n_negative": 155,
+      "roc_auc": 0.996973317403425
+    }
+  },
+  "per_source": {
+    "TREC-06": {
+      "threshold": 0.5,
+      "accuracy": 0.9478591583854742,
+      "precision": 0.8307257714037375,
+      "recall": 0.9817668207498716,
+      "f1": 0.8999529190207156,
+      "specificity": 0.9372179239200515,
+      "false_positive_rate": 0.06278207607994842,
+      "false_negative_rate": 0.018233179250128403,
+      "confusion_matrix": {
+        "tn": 11629,
+        "fp": 779,
+        "fn": 71,
+        "tp": 3823
+      },
+      "n": 16302,
+      "n_positive": 3894,
+      "n_negative": 12408,
+      "roc_auc": 0.9942507931824557,
+      "label3_counts": {
+        "legit": 12408,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 3894
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 3823,
+        "mean_tokens_full": 284.4,
+        "mean_tokens_fed": 232.2,
+        "median_tokens_full": 207.0,
+        "pct_truncated": 15.3
+      },
+      "tn": {
+        "n": 11629,
+        "mean_tokens_full": 781.6,
+        "mean_tokens_fed": 294.6,
+        "median_tokens_full": 269.0,
+        "pct_truncated": 23.6
+      },
+      "fp": {
+        "n": 779,
+        "mean_tokens_full": 1778.7,
+        "mean_tokens_fed": 280.1,
+        "median_tokens_full": 256.0,
+        "pct_truncated": 30.8
+      },
+      "fn": {
+        "n": 71,
+        "mean_tokens_full": 309.8,
+        "mean_tokens_fed": 215.2,
+        "median_tokens_full": 181.0,
+        "pct_truncated": 15.5
+      },
+      "correct": {
+        "n": 15452,
+        "mean_tokens_full": 658.6,
+        "mean_tokens_fed": 279.1,
+        "median_tokens_full": 246.0,
+        "pct_truncated": 21.5
+      },
+      "wrong": {
+        "n": 850,
+        "mean_tokens_full": 1656.0,
+        "mean_tokens_fed": 274.7,
+        "median_tokens_full": 229.5,
+        "pct_truncated": 29.5
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 1363,
+        "n_positive": 587,
+        "positive_rate": 0.43066764490095377,
+        "recall": 0.9863713798977853,
+        "fpr": 0.17525773195876287,
+        "accuracy": 0.8943506969919296
+      },
+      {
+        "bucket": "65-128",
+        "n": 2538,
+        "n_positive": 736,
+        "positive_rate": 0.2899921197793538,
+        "recall": 0.96875,
+        "fpr": 0.06714761376248612,
+        "accuracy": 0.9432624113475178
+      },
+      {
+        "bucket": "129-256",
+        "n": 4594,
+        "n_positive": 1196,
+        "positive_rate": 0.260339573356552,
+        "recall": 0.9816053511705686,
+        "fpr": 0.039140670982931135,
+        "accuracy": 0.9662603395733566
+      },
+      {
+        "bucket": "257-512",
+        "n": 4232,
+        "n_positive": 780,
+        "positive_rate": 0.1843100189035917,
+        "recall": 0.9910256410256411,
+        "fpr": 0.04316338354577057,
+        "accuracy": 0.9631379962192816
+      },
+      {
+        "bucket": ">512",
+        "n": 3575,
+        "n_positive": 595,
+        "positive_rate": 0.16643356643356644,
+        "recall": 0.9815126050420168,
+        "fpr": 0.08053691275167785,
+        "accuracy": 0.9297902097902098
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 16302,
+    "seconds": 57.53,
+    "rows_per_s": 283.4,
+    "padded_tokens": 4606544,
+    "real_tokens": 4546874,
+    "padded_tokens_per_s": 80074,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 10.12,
+      "p50_ms": 8.95,
+      "p95_ms": 13.37,
+      "max_ms": 19.01,
+      "avg_s": 0.0101,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.137,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 411.57,
+      "p50_ms": 397.57,
+      "p95_ms": 630.3,
+      "max_ms": 674.98,
+      "avg_s": 0.4116,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 5.562,
+      "threads": 16,
+      "estimated_full_run_s": 38.4
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-weighted/best",
+    "run_name": "ft-weighted",
+    "params": 337444867,
+    "timestamp": "2026-10-09 05:55",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-weighted/error_analysis.md",
+    "n_fn": 71,
+    "n_fp": 779,
+    "composition": {
+      "fp": {
+        "n": 779,
+        "mailing_list_or_digest": 0.225,
+        "html_heavy": 0.001,
+        "very_short_lt20_tokens": 0.026,
+        "truncated_gt512": 0.308,
+        "lure_phrase": 0.014
+      },
+      "tn": {
+        "n": 11629,
+        "mailing_list_or_digest": 0.297,
+        "html_heavy": 0.002,
+        "very_short_lt20_tokens": 0.004,
+        "truncated_gt512": 0.236,
+        "lure_phrase": 0.012
+      },
+      "fn": {
+        "n": 71,
+        "mailing_list_or_digest": 0.056,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.155,
+        "lure_phrase": 0.014
+      },
+      "tp": {
+        "n": 3823,
+        "mailing_list_or_digest": 0.029,
+        "html_heavy": 0.001,
+        "very_short_lt20_tokens": 0.002,
+        "truncated_gt512": 0.153,
+        "lure_phrase": 0.063
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "blocki",
+        "share_a": 0.169,
+        "share_b": 0.0,
+        "ratio": 1336.89
+      },
+      {
+        "token": "scanmail",
+        "share_a": 0.169,
+        "share_b": 0.0,
+        "ratio": 1336.89
+      },
+      {
+        "token": "smailex",
+        "share_a": 0.169,
+        "share_b": 0.0,
+        "ratio": 1336.89
+      },
+      {
+        "token": "progra",
+        "share_a": 0.169,
+        "share_b": 0.0,
+        "ratio": 1336.89
+      },
+      {
+        "token": "scanning",
+        "share_a": 0.169,
+        "share_b": 0.0,
+        "ratio": 1336.89
+      },
+      {
+        "token": "matched",
+        "share_a": 0.169,
+        "share_b": 0.0,
+        "ratio": 445.63
+      },
+      {
+        "token": "kool",
+        "share_a": 0.042,
+        "share_b": 0.0,
+        "ratio": 374.33
+      },
+      {
+        "token": "tex-fonts",
+        "share_a": 0.113,
+        "share_b": 0.0,
+        "ratio": 303.03
+      },
+      {
+        "token": "blocking",
+        "share_a": 0.183,
+        "share_b": 0.001,
+        "ratio": 288.77
+      },
+      {
+        "token": "pattern",
+        "share_a": 0.183,
+        "share_b": 0.001,
+        "ratio": 288.77
+      },
+      {
+        "token": "exe",
+        "share_a": 0.056,
+        "share_b": 0.0,
+        "ratio": 160.43
+      },
+      {
+        "token": "setting",
+        "share_a": 0.183,
+        "share_b": 0.001,
+        "ratio": 131.26
+      },
+      {
+        "token": "overbehitchreveiw",
+        "share_a": 0.042,
+        "share_b": 0.0,
+        "ratio": 124.78
+      },
+      {
+        "token": "tex-font",
+        "share_a": 0.056,
+        "share_b": 0.001,
+        "ratio": 96.26
+      },
+      {
+        "token": "utah",
+        "share_a": 0.169,
+        "share_b": 0.002,
+        "ratio": 89.13
+      },
+      {
+        "token": "detected",
+        "share_a": 0.169,
+        "share_b": 0.002,
+        "ratio": 89.13
+      },
+      {
+        "token": "unhappy",
+        "share_a": 0.042,
+        "share_b": 0.001,
+        "ratio": 74.87
+      },
+      {
+        "token": "funny",
+        "share_a": 0.042,
+        "share_b": 0.001,
+        "ratio": 74.87
+      },
+      {
+        "token": "mobitel0",
+        "share_a": 0.042,
+        "share_b": 0.001,
+        "ratio": 74.87
+      },
+      {
+        "token": "mobitel",
+        "share_a": 0.042,
+        "share_b": 0.001,
+        "ratio": 74.87
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "referring",
+        "share_a": 0.05,
+        "share_b": 0.007,
+        "ratio": 7.6
+      },
+      {
+        "token": "https",
+        "share_a": 0.049,
+        "share_b": 0.007,
+        "ratio": 6.72
+      },
+      {
+        "token": "inform",
+        "share_a": 0.056,
+        "share_b": 0.009,
+        "ratio": 6.54
+      },
+      {
+        "token": "forwarding",
+        "share_a": 0.09,
+        "share_b": 0.014,
+        "ratio": 6.32
+      },
+      {
+        "token": "coordinator",
+        "share_a": 0.06,
+        "share_b": 0.01,
+        "ratio": 6.03
+      },
+      {
+        "token": "todd",
+        "share_a": 0.049,
+        "share_b": 0.008,
+        "ratio": 6.01
+      },
+      {
+        "token": "official",
+        "share_a": 0.062,
+        "share_b": 0.012,
+        "ratio": 5.26
+      },
+      {
+        "token": "esmtp",
+        "share_a": 0.074,
+        "share_b": 0.014,
+        "ratio": 5.18
+      },
+      {
+        "token": "brought",
+        "share_a": 0.054,
+        "share_b": 0.011,
+        "ratio": 5.13
+      },
+      {
+        "token": "safe",
+        "share_a": 0.059,
+        "share_b": 0.012,
+        "ratio": 5.12
+      },
+      {
+        "token": "precedence",
+        "share_a": 0.05,
+        "share_b": 0.01,
+        "ratio": 4.97
+      },
+      {
+        "token": "hotmail",
+        "share_a": 0.059,
+        "share_b": 0.012,
+        "ratio": 4.9
+      },
+      {
+        "token": "friends",
+        "share_a": 0.071,
+        "share_b": 0.015,
+        "ratio": 4.66
+      },
+      {
+        "token": "private",
+        "share_a": 0.071,
+        "share_b": 0.016,
+        "ratio": 4.49
+      },
+      {
+        "token": "mime-version",
+        "share_a": 0.081,
+        "share_b": 0.018,
+        "ratio": 4.46
+      },
+      {
+        "token": "locate",
+        "share_a": 0.051,
+        "share_b": 0.012,
+        "ratio": 4.46
+      },
+      {
+        "token": "content-transfer-encoding",
+        "share_a": 0.065,
+        "share_b": 0.015,
+        "ratio": 4.4
+      },
+      {
+        "token": "decide",
+        "share_a": 0.056,
+        "share_b": 0.013,
+        "ratio": 4.38
+      },
+      {
+        "token": "sender",
+        "share_a": 0.082,
+        "share_b": 0.019,
+        "ratio": 4.36
+      },
+      {
+        "token": "international",
+        "share_a": 0.087,
+        "share_b": 0.021,
+        "ratio": 4.2
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:052869",
+      "primary:020947",
+      "primary:121268",
+      "primary:086987",
+      "primary:122099",
+      "primary:161198",
+      "primary:044992",
+      "primary:145723",
+      "primary:056767",
+      "primary:129745",
+      "primary:147887",
+      "primary:173938",
+      "primary:173636",
+      "primary:160104",
+      "primary:024565",
+      "primary:063077",
+      "primary:072993",
+      "primary:017595",
+      "primary:080991",
+      "primary:076023",
+      "primary:030013",
+      "primary:186523",
+      "primary:154489",
+      "primary:071577",
+      "primary:143065"
+    ],
+    "top_fp_doc_ids": [
+      "primary:174983",
+      "primary:118028",
+      "primary:143547",
+      "primary:182736",
+      "primary:070866",
+      "primary:040048",
+      "primary:049322",
+      "primary:011014",
+      "primary:051867",
+      "primary:032159",
+      "primary:015304",
+      "primary:092217",
+      "primary:052736",
+      "primary:030534",
+      "primary:158244",
+      "primary:066681",
+      "primary:202546",
+      "primary:180285",
+      "primary:129925",
+      "primary:011118",
+      "primary:189627",
+      "primary:107132",
+      "primary:194063",
+      "primary:057266",
+      "primary:170605"
+    ],
+    "threeway": null
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-weighted/confusion_matrix.png",
+    "roc_curve": "reports/eval_ft-weighted/roc_curve.png",
+    "pr_curve": "reports/eval_ft-weighted/pr_curve.png"
+  }
+}
diff --git a/presentation/reports/eval_ft-weighted/metrics_test3.json b/presentation/reports/eval_ft-weighted/metrics_test3.json
new file mode 100644
index 0000000000000000000000000000000000000000..eb9e905d986e83541e7bea7f7d02a85baf4d511c
--- /dev/null
+++ b/presentation/reports/eval_ft-weighted/metrics_test3.json
@@ -0,0 +1,1669 @@
+{
+  "timestamp": "2026-10-09 05:55",
+  "run_name": "ft-weighted",
+  "split": "test3",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout3.parquet",
+  "split_title": "three-way holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-weighted/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-weighted",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "weighted_cross_entropy",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 1,
+    "best_step": 10000,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.99373795761079,
+    "val_metrics": {
+      "accuracy": 0.9931954985605862,
+      "precision": 0.9911121787172712,
+      "recall": 0.9963776865491427,
+      "f1": 0.99373795761079,
+      "roc_auc": 0.9997532357160727,
+      "pr_auc": 0.9997920087331971,
+      "tn": 6928,
+      "fp": 74,
+      "fn": 30,
+      "tp": 8252,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.8991281390190125,
+      "f1_at_optimal_threshold": 0.9942622455758894,
+      "f1_unwanted": 0.99373795761079,
+      "n_3way": 8105,
+      "accuracy_3way": 0.9902529302899444,
+      "macro_f1_3way": 0.9774841428658321,
+      "precision_legitimate": 0.9992789154888952,
+      "recall_legitimate": 0.9895744073121965,
+      "f1_legitimate": 0.9944029850746269,
+      "precision_phishing": 0.9630681818181818,
+      "recall_phishing": 0.9970588235294118,
+      "f1_phishing": 0.9797687861271677,
+      "precision_spam": 0.9255189255189256,
+      "recall_spam": 0.9934469200524246,
+      "f1_spam": 0.9582806573957017,
+      "cm3_ll": 6929,
+      "cm3_lp": 12,
+      "cm3_ls": 61,
+      "cm3_pl": 1,
+      "cm3_pp": 339,
+      "cm3_ps": 0,
+      "cm3_sl": 4,
+      "cm3_sp": 1,
+      "cm3_ss": 758,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 339,
+      "pvs_phish_as_spam": 1,
+      "pvs_spam_as_phish": 2,
+      "pvs_spam_as_spam": 761,
+      "val_loss": 0.025379083498832725,
+      "n": 15284,
+      "eval_seconds": 47.8
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9931954985605862,
+        "precision": 0.9911121787172712,
+        "recall": 0.9963776865491427,
+        "f1": 0.99373795761079,
+        "roc_auc": 0.9997532357160727,
+        "pr_auc": 0.9997920087331971,
+        "tn": 6928,
+        "fp": 74,
+        "fn": 30,
+        "tp": 8252,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.8991281390190125,
+        "f1_at_optimal_threshold": 0.9942622455758894,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.9902529302899444,
+        "macro_f1": 0.9774841428658321,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9992789154888952,
+            "recall": 0.9895744073121965,
+            "f1": 0.9944029850746269
+          },
+          "Phishing": {
+            "precision": 0.9630681818181818,
+            "recall": 0.9970588235294118,
+            "f1": 0.9797687861271677
+          },
+          "Spam": {
+            "precision": 0.9255189255189256,
+            "recall": 0.9934469200524246,
+            "f1": 0.9582806573957017
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6929,
+              12,
+              61
+            ],
+            [
+              1,
+              339,
+              0
+            ],
+            [
+              4,
+              1,
+              758
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              339,
+              1
+            ],
+            [
+              2,
+              761
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.8991281390190125,
+      "val_f1_at_optimal": 0.9942622455758894
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 3220,
+    "positive": 1221,
+    "negative": 1999,
+    "positive_rate": 0.3792,
+    "label3": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "by_source": {
+      "Assassin": {
+        "rows": 236,
+        "positive": 72,
+        "label3": {
+          "legit": 164,
+          "phish": 0,
+          "spam": 72,
+          "unwanted": 0
+        }
+      },
+      "CEAS-08": {
+        "rows": 736,
+        "positive": 0,
+        "label3": {
+          "legit": 736,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Enron": {
+        "rows": 628,
+        "positive": 628,
+        "label3": {
+          "legit": 0,
+          "phish": 0,
+          "spam": 628,
+          "unwanted": 0
+        }
+      },
+      "Nazario": {
+        "rows": 222,
+        "positive": 222,
+        "label3": {
+          "legit": 0,
+          "phish": 222,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Nigerian_Fraud": {
+        "rows": 299,
+        "positive": 299,
+        "label3": {
+          "legit": 0,
+          "phish": 299,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "TREC-07": {
+        "rows": 1099,
+        "positive": 0,
+        "label3": {
+          "legit": 1099,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      }
+    },
+    "mean_tokens_full": 835.2,
+    "pct_truncated": 34.01
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9894409937888199,
+    "precision": 0.9790153349475383,
+    "recall": 0.9934479934479934,
+    "f1": 0.9861788617886179,
+    "specificity": 0.9869934967483742,
+    "false_positive_rate": 0.013006503251625813,
+    "false_negative_rate": 0.006552006552006552,
+    "confusion_matrix": {
+      "tn": 1973,
+      "fp": 26,
+      "fn": 8,
+      "tp": 1213
+    },
+    "n": 3220,
+    "n_positive": 1221,
+    "n_negative": 1999
+  },
+  "threshold_free": {
+    "roc_auc": 0.999348773485842,
+    "pr_auc": 0.9988435835907616,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.990990990990991,
+      "fpr": 0.009504752376188095,
+      "threshold": 0.7280054092407227,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.7059787059787059,
+      "fpr": 0.0005002501250625312,
+      "threshold": 0.9999038577079773,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.7280054092407227,
+      "f1": 0.9877551020408163,
+      "precision": 0.9845402766476811,
+      "recall": 0.990990990990991
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 1213,
+        "predicted_phishing": 515,
+        "predicted_spam": 698
+      },
+      "false_positives_flagged": {
+        "n": 26,
+        "predicted_phishing": 5,
+        "predicted_spam": 21
+      },
+      "missed_positives": {
+        "n": 8,
+        "predicted_phishing": 1,
+        "predicted_spam": 7
+      },
+      "all_flagged": {
+        "n": 1239,
+        "predicted_phishing": 520,
+        "predicted_spam": 719
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 0,
+        "predicted_phishing": 0,
+        "predicted_spam": 0
+      }
+    },
+    "rows_with_full_label": 3220,
+    "rows_unwanted_only": 0,
+    "evaluated": true,
+    "reason": null,
+    "confusion_matrix": {
+      "labels": [
+        "Legitimate",
+        "Phishing",
+        "Spam"
+      ],
+      "matrix": [
+        [
+          1973,
+          5,
+          21
+        ],
+        [
+          2,
+          514,
+          5
+        ],
+        [
+          6,
+          1,
+          693
+        ]
+      ],
+      "cells": {
+        "true_legitimate": {
+          "pred_legitimate": 1973,
+          "pred_phishing": 5,
+          "pred_spam": 21
+        },
+        "true_phishing": {
+          "pred_legitimate": 2,
+          "pred_phishing": 514,
+          "pred_spam": 5
+        },
+        "true_spam": {
+          "pred_legitimate": 6,
+          "pred_phishing": 1,
+          "pred_spam": 693
+        }
+      }
+    },
+    "accuracy": 0.9875776397515528,
+    "macro_f1": 0.985237826721197,
+    "macro_f1_present_classes": 0.985237826721197,
+    "classes_present": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "per_class": {
+      "Legitimate": {
+        "precision": 0.9959616355376073,
+        "recall": 0.9869934967483742,
+        "f1": 0.9914572864321608,
+        "support": 1999
+      },
+      "Phishing": {
+        "precision": 0.9884615384615385,
+        "recall": 0.9865642994241842,
+        "f1": 0.9875120076849183,
+        "support": 521
+      },
+      "Spam": {
+        "precision": 0.9638386648122392,
+        "recall": 0.99,
+        "f1": 0.9767441860465116,
+        "support": 700
+      }
+    },
+    "argmax_rule": {
+      "doc": "plain argmax over the three probabilities (secondary; can disagree with the binary verdict near the threshold)",
+      "accuracy": 0.987888198757764,
+      "macro_f1": 0.9855518791290724,
+      "per_class": {
+        "Legitimate": {
+          "precision": 0.9959636730575177,
+          "recall": 0.9874937468734367,
+          "f1": 0.9917106254709872,
+          "support": 1999
+        },
+        "Phishing": {
+          "precision": 0.9884615384615385,
+          "recall": 0.9865642994241842,
+          "f1": 0.9875120076849183,
+          "support": 521
+        },
+        "Spam": {
+          "precision": 0.9651810584958217,
+          "recall": 0.99,
+          "f1": 0.9774330042313117,
+          "support": 700
+        }
+      },
+      "rows_where_rules_differ": 1
+    },
+    "phish_vs_spam_on_true_unwanted": {
+      "n_true_unwanted": 1221,
+      "kind_only": {
+        "doc": "given the row is unwanted, is argmax(P(phishing), P(spam)) right? (independent of the roll-up threshold)",
+        "labels": [
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            515,
+            6
+          ],
+          [
+            1,
+            699
+          ]
+        ],
+        "cells": {
+          "true_phishing": {
+            "pred_phishing": 515,
+            "pred_spam": 6
+          },
+          "true_spam": {
+            "pred_phishing": 1,
+            "pred_spam": 699
+          }
+        },
+        "accuracy": 0.9942669942669943,
+        "phishing_as_spam": 6,
+        "spam_as_phishing": 1
+      },
+      "rollup_rule": {
+        "doc": "rows = true Phishing / Spam; columns = predicted Legitimate (missed) / Phishing / Spam under the roll-up rule",
+        "rows": [
+          "Phishing",
+          "Spam"
+        ],
+        "columns": [
+          "Legitimate",
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            2,
+            514,
+            5
+          ],
+          [
+            6,
+            1,
+            693
+          ]
+        ],
+        "missed_as_legitimate": {
+          "phishing": 2,
+          "spam": 6
+        }
+      }
+    },
+    "per_source": {
+      "Assassin": {
+        "n": 236,
+        "counts": {
+          "Legitimate": 164,
+          "Phishing": 0,
+          "Spam": 72
+        },
+        "accuracy": 0.961864406779661,
+        "macro_f1": 0.6401713738091211,
+        "macro_f1_present_classes": 0.9602570607136818,
+        "per_class_recall": {
+          "Legitimate": 0.9634146341463414,
+          "Spam": 0.9583333333333334
+        }
+      },
+      "CEAS-08": {
+        "n": 736,
+        "counts": {
+          "Legitimate": 736,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9823369565217391,
+        "macro_f1": 0.3303632625085675,
+        "macro_f1_present_classes": 0.9910897875257025,
+        "per_class_recall": {
+          "Legitimate": 0.9823369565217391
+        }
+      },
+      "Enron": {
+        "n": 628,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 0,
+          "Spam": 628
+        },
+        "accuracy": 0.9936305732484076,
+        "macro_f1": 0.33226837060702874,
+        "macro_f1_present_classes": 0.9968051118210862,
+        "per_class_recall": {
+          "Spam": 0.9936305732484076
+        }
+      },
+      "Nazario": {
+        "n": 222,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 222,
+          "Spam": 0
+        },
+        "accuracy": 0.9819819819819819,
+        "macro_f1": 0.3303030303030303,
+        "macro_f1_present_classes": 0.990909090909091,
+        "per_class_recall": {
+          "Phishing": 0.9819819819819819
+        }
+      },
+      "Nigerian_Fraud": {
+        "n": 299,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 299,
+          "Spam": 0
+        },
+        "accuracy": 0.9899665551839465,
+        "macro_f1": 0.33165266106442576,
+        "macro_f1_present_classes": 0.9949579831932773,
+        "per_class_recall": {
+          "Phishing": 0.9899665551839465
+        }
+      },
+      "TREC-07": {
+        "n": 1099,
+        "counts": {
+          "Legitimate": 1099,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9936305732484076,
+        "macro_f1": 0.33226837060702874,
+        "macro_f1_present_classes": 0.9968051118210862,
+        "per_class_recall": {
+          "Legitimate": 0.9936305732484076
+        }
+      }
+    }
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 139,
+      "bank account": 91,
+      "urgently": 25,
+      "verify your account": 15,
+      "confirm your account": 9,
+      "update your account": 9,
+      "within 24 hours": 8,
+      "security alert": 8,
+      "paypal account": 7,
+      "password will expire": 7,
+      "click here to verify": 6,
+      "unusual activity": 4,
+      "wire transfer": 3,
+      "update your information": 3,
+      "gift card": 3,
+      "verify your identity": 2,
+      "unusual sign-in": 2,
+      "account will be suspended": 2,
+      "update your payment": 2,
+      "failure to verify": 2,
+      "within 48 hours": 2,
+      "confirm your identity": 2,
+      "enter your password": 1,
+      "update your billing": 1,
+      "reset your password": 1,
+      "suspicious activity": 1,
+      "security notice": 1,
+      "log in to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 15,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "update your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 2,
+        "negative": 1
+      },
+      "update your billing": {
+        "positive": 1,
+        "negative": 0
+      },
+      "update your payment": {
+        "positive": 2,
+        "negative": 0
+      },
+      "click here to verify": {
+        "positive": 6,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual sign-in": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "suspicious activity": {
+        "positive": 1,
+        "negative": 0
+      },
+      "paypal account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "password will expire": {
+        "positive": 7,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 1,
+        "negative": 0
+      },
+      "wire transfer": {
+        "positive": 2,
+        "negative": 1
+      },
+      "gift card": {
+        "positive": 3,
+        "negative": 0
+      },
+      "urgent": {
+        "positive": 133,
+        "negative": 6
+      },
+      "urgently": {
+        "positive": 25,
+        "negative": 0
+      },
+      "security alert": {
+        "positive": 8,
+        "negative": 0
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 91,
+        "negative": 0
+      },
+      "failure to verify": {
+        "positive": 2,
+        "negative": 0
+      },
+      "within 24 hours": {
+        "positive": 8,
+        "negative": 0
+      },
+      "within 48 hours": {
+        "positive": 2,
+        "negative": 0
+      }
+    },
+    "label_purity": 0.9745,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 7,
+      "phish": 236,
+      "spam": 32,
+      "unwanted": 0
+    },
+    "threeway_accuracy": 0.9927272727272727,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 1.0,
+      "precision": 1.0,
+      "recall": 1.0,
+      "f1": 1.0,
+      "specificity": 1.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 7,
+        "fp": 0,
+        "fn": 0,
+        "tp": 268
+      },
+      "n": 275,
+      "n_positive": 268,
+      "n_negative": 7,
+      "roc_auc": 1.0
+    }
+  },
+  "per_source": {
+    "Assassin": {
+      "threshold": 0.5,
+      "accuracy": 0.9661016949152542,
+      "precision": 0.9210526315789473,
+      "recall": 0.9722222222222222,
+      "f1": 0.9459459459459458,
+      "specificity": 0.9634146341463414,
+      "false_positive_rate": 0.036585365853658534,
+      "false_negative_rate": 0.027777777777777776,
+      "confusion_matrix": {
+        "tn": 158,
+        "fp": 6,
+        "fn": 2,
+        "tp": 70
+      },
+      "n": 236,
+      "n_positive": 72,
+      "n_negative": 164,
+      "roc_auc": 0.9953421409214093,
+      "label3_counts": {
+        "legit": 164,
+        "phish": 0,
+        "spam": 72,
+        "unwanted": 0
+      }
+    },
+    "CEAS-08": {
+      "threshold": 0.5,
+      "accuracy": 0.9823369565217391,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9823369565217391,
+      "false_positive_rate": 0.017663043478260868,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 723,
+        "fp": 13,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 736,
+      "n_positive": 0,
+      "n_negative": 736,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 736,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Enron": {
+      "threshold": 0.5,
+      "accuracy": 0.9936305732484076,
+      "precision": 1.0,
+      "recall": 0.9936305732484076,
+      "f1": 0.9968051118210862,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.006369426751592357,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 4,
+        "tp": 624
+      },
+      "n": 628,
+      "n_positive": 628,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 0,
+        "spam": 628,
+        "unwanted": 0
+      }
+    },
+    "Nazario": {
+      "threshold": 0.5,
+      "accuracy": 0.990990990990991,
+      "precision": 1.0,
+      "recall": 0.990990990990991,
+      "f1": 0.9954751131221719,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.009009009009009009,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 2,
+        "tp": 220
+      },
+      "n": 222,
+      "n_positive": 222,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 222,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Nigerian_Fraud": {
+      "threshold": 0.5,
+      "accuracy": 1.0,
+      "precision": 1.0,
+      "recall": 1.0,
+      "f1": 1.0,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 0,
+        "tp": 299
+      },
+      "n": 299,
+      "n_positive": 299,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 299,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "TREC-07": {
+      "threshold": 0.5,
+      "accuracy": 0.9936305732484076,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9936305732484076,
+      "false_positive_rate": 0.006369426751592357,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 1092,
+        "fp": 7,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 1099,
+      "n_positive": 0,
+      "n_negative": 1099,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 1099,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 1213,
+        "mean_tokens_full": 994.5,
+        "mean_tokens_fed": 279.6,
+        "median_tokens_full": 241.0,
+        "pct_truncated": 26.6
+      },
+      "tn": {
+        "n": 1973,
+        "mean_tokens_full": 729.2,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.3
+      },
+      "fp": {
+        "n": 26,
+        "mean_tokens_full": 1537.1,
+        "mean_tokens_fed": 356.2,
+        "median_tokens_full": 706.0,
+        "pct_truncated": 53.8
+      },
+      "fn": {
+        "n": 8,
+        "mean_tokens_full": 550.4,
+        "mean_tokens_fed": 326.4,
+        "median_tokens_full": 334.5,
+        "pct_truncated": 37.5
+      },
+      "correct": {
+        "n": 3186,
+        "mean_tokens_full": 830.2,
+        "mean_tokens_fed": 328.5,
+        "median_tokens_full": 338.0,
+        "pct_truncated": 33.8
+      },
+      "wrong": {
+        "n": 34,
+        "mean_tokens_full": 1304.9,
+        "mean_tokens_fed": 349.1,
+        "median_tokens_full": 485.5,
+        "pct_truncated": 50.0
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "unwanted": {
+        "n": 1221,
+        "mean_tokens_full": 991.6,
+        "mean_tokens_fed": 279.9,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.7
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "phish": {
+        "n": 521,
+        "mean_tokens_full": 1349.4,
+        "mean_tokens_fed": 355.1,
+        "median_tokens_full": 412.0,
+        "pct_truncated": 39.9
+      },
+      "spam": {
+        "n": 700,
+        "mean_tokens_full": 725.3,
+        "mean_tokens_fed": 223.9,
+        "median_tokens_full": 159.0,
+        "pct_truncated": 16.9
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 129,
+        "n_positive": 99,
+        "positive_rate": 0.7674418604651163,
+        "recall": 0.98989898989899,
+        "fpr": 0.1,
+        "accuracy": 0.9689922480620154
+      },
+      {
+        "bucket": "65-128",
+        "n": 427,
+        "n_positive": 252,
+        "positive_rate": 0.5901639344262295,
+        "recall": 0.996031746031746,
+        "fpr": 0.011428571428571429,
+        "accuracy": 0.9929742388758782
+      },
+      {
+        "bucket": "129-256",
+        "n": 699,
+        "n_positive": 276,
+        "positive_rate": 0.3948497854077253,
+        "recall": 1.0,
+        "fpr": 0.0070921985815602835,
+        "accuracy": 0.9957081545064378
+      },
+      {
+        "bucket": "257-512",
+        "n": 870,
+        "n_positive": 268,
+        "positive_rate": 0.3080459770114943,
+        "recall": 0.9888059701492538,
+        "fpr": 0.006644518272425249,
+        "accuracy": 0.9919540229885058
+      },
+      {
+        "bucket": ">512",
+        "n": 1095,
+        "n_positive": 326,
+        "positive_rate": 0.29771689497716897,
+        "recall": 0.99079754601227,
+        "fpr": 0.018205461638491547,
+        "accuracy": 0.9844748858447488
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 3220,
+    "seconds": 13.51,
+    "rows_per_s": 238.3,
+    "padded_tokens": 1079936,
+    "real_tokens": 1058490,
+    "padded_tokens_per_s": 79930,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 10.12,
+      "p50_ms": 8.95,
+      "p95_ms": 13.37,
+      "max_ms": 19.01,
+      "avg_s": 0.0101,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.137,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 411.57,
+      "p50_ms": 397.57,
+      "p95_ms": 630.3,
+      "max_ms": 674.98,
+      "avg_s": 0.4116,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 5.562,
+      "threads": 16,
+      "estimated_full_run_s": 38.4
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-weighted/best",
+    "run_name": "ft-weighted",
+    "params": 337444867,
+    "timestamp": "2026-10-09 05:55",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "reports/eval_ft-weighted/error_analysis_test3.md",
+    "n_fn": 8,
+    "n_fp": 26,
+    "composition": {
+      "fp": {
+        "n": 26,
+        "mailing_list_or_digest": 0.577,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.538,
+        "lure_phrase": 0.0
+      },
+      "tn": {
+        "n": 1973,
+        "mailing_list_or_digest": 0.563,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.383,
+        "lure_phrase": 0.004
+      },
+      "fn": {
+        "n": 8,
+        "mailing_list_or_digest": 0.25,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.375,
+        "lure_phrase": 0.0
+      },
+      "tp": {
+        "n": 1213,
+        "mailing_list_or_digest": 0.086,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.007,
+        "truncated_gt512": 0.266,
+        "lure_phrase": 0.221
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "invite",
+        "share_a": 0.5,
+        "share_b": 0.002,
+        "ratio": 183.55
+      },
+      {
+        "token": "networking",
+        "share_a": 0.375,
+        "share_b": 0.003,
+        "ratio": 111.04
+      },
+      {
+        "token": "event",
+        "share_a": 0.5,
+        "share_b": 0.005,
+        "ratio": 98.84
+      },
+      {
+        "token": "angeles",
+        "share_a": 0.375,
+        "share_b": 0.004,
+        "ratio": 90.85
+      },
+      {
+        "token": "york",
+        "share_a": 0.375,
+        "share_b": 0.005,
+        "ratio": 76.87
+      },
+      {
+        "token": "technologies",
+        "share_a": 0.375,
+        "share_b": 0.006,
+        "ratio": 66.62
+      },
+      {
+        "token": "entertainment",
+        "share_a": 0.375,
+        "share_b": 0.007,
+        "ratio": 58.79
+      },
+      {
+        "token": "association",
+        "share_a": 0.375,
+        "share_b": 0.008,
+        "ratio": 47.59
+      },
+      {
+        "token": "hosting",
+        "share_a": 0.375,
+        "share_b": 0.009,
+        "ratio": 43.45
+      },
+      {
+        "token": "staff",
+        "share_a": 0.375,
+        "share_b": 0.012,
+        "ratio": 34.46
+      },
+      {
+        "token": "changes",
+        "share_a": 0.375,
+        "share_b": 0.012,
+        "ratio": 32.24
+      },
+      {
+        "token": "media",
+        "share_a": 0.375,
+        "share_b": 0.015,
+        "ratio": 27.01
+      },
+      {
+        "token": "cards",
+        "share_a": 0.375,
+        "share_b": 0.015,
+        "ratio": 27.01
+      },
+      {
+        "token": "leading",
+        "share_a": 0.375,
+        "share_b": 0.015,
+        "ratio": 27.01
+      },
+      {
+        "token": "los",
+        "share_a": 0.375,
+        "share_b": 0.016,
+        "ratio": 25.62
+      },
+      {
+        "token": "anyone",
+        "share_a": 0.375,
+        "share_b": 0.017,
+        "ratio": 23.24
+      },
+      {
+        "token": "simply",
+        "share_a": 0.5,
+        "share_b": 0.025,
+        "ratio": 21.06
+      },
+      {
+        "token": "browser",
+        "share_a": 0.5,
+        "share_b": 0.026,
+        "ratio": 19.77
+      },
+      {
+        "token": "friends",
+        "share_a": 0.375,
+        "share_b": 0.021,
+        "ratio": 18.86
+      },
+      {
+        "token": "paste",
+        "share_a": 0.375,
+        "share_b": 0.021,
+        "ratio": 18.86
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "zzz-unspun",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 223.42
+      },
+      {
+        "token": "czzz-unspun",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 223.42
+      },
+      {
+        "token": "outlets",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 223.42
+      },
+      {
+        "token": "cowpokeproductions",
+        "share_a": 0.115,
+        "share_b": 0.001,
+        "ratio": 173.77
+      },
+      {
+        "token": "trinity",
+        "share_a": 0.115,
+        "share_b": 0.001,
+        "ratio": 173.77
+      },
+      {
+        "token": "cowpoke",
+        "share_a": 0.115,
+        "share_b": 0.001,
+        "ratio": 173.77
+      },
+      {
+        "token": "enron",
+        "share_a": 0.115,
+        "share_b": 0.001,
+        "ratio": 173.77
+      },
+      {
+        "token": "full-color",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 138.3
+      },
+      {
+        "token": "vineyard",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "patel",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "loripatel",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "englishmajor",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "comtm",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "asbrand",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "spins",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "unspun",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "savviest",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "mediaunspun",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "four-week",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      },
+      {
+        "token": "authoritatively",
+        "share_a": 0.154,
+        "share_b": 0.001,
+        "ratio": 134.05
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:148887",
+      "primary:191966",
+      "aura:Nazario:000643",
+      "primary:007176",
+      "aura:Nazario:000097",
+      "primary:001939",
+      "primary:046569",
+      "primary:014300"
+    ],
+    "top_fp_doc_ids": [
+      "primary:091801",
+      "primary:144048",
+      "primary:199386",
+      "primary:103927",
+      "primary:055965",
+      "primary:165228",
+      "primary:152763",
+      "primary:048756",
+      "primary:168120",
+      "primary:041713",
+      "primary:069336",
+      "primary:057514",
+      "primary:146259",
+      "primary:156887",
+      "primary:116110",
+      "primary:132924",
+      "primary:156985",
+      "primary:115649",
+      "primary:153841",
+      "primary:117398",
+      "primary:021911",
+      "primary:003448",
+      "primary:025142",
+      "primary:113001",
+      "primary:027029"
+    ],
+    "threeway": {
+      "n_phish_as_spam": 5,
+      "n_spam_as_phish": 1,
+      "top_phish_as_spam_doc_ids": [
+        "aura:Nigerian_Fraud:001698",
+        "aura:Nigerian_Fraud:000337",
+        "aura:Nigerian_Fraud:003049",
+        "aura:Nazario:001179",
+        "aura:Nazario:000532"
+      ],
+      "top_spam_as_phish_doc_ids": [
+        "primary:129677"
+      ],
+      "phish_as_spam_tokens": [
+        {
+          "token": "friends",
+          "share_a": 0.6,
+          "share_b": 0.016,
+          "ratio": 38.52
+        },
+        {
+          "token": "join",
+          "share_a": 0.6,
+          "share_b": 0.021,
+          "ratio": 28.47
+        },
+        {
+          "token": "complete",
+          "share_a": 0.6,
+          "share_b": 0.049,
+          "ratio": 12.84
+        },
+        {
+          "token": "don't",
+          "share_a": 0.8,
+          "share_b": 0.068,
+          "ratio": 11.86
+        },
+        {
+          "token": "visit",
+          "share_a": 0.6,
+          "share_b": 0.058,
+          "ratio": 10.73
+        },
+        {
+          "token": "every",
+          "share_a": 0.6,
+          "share_b": 0.068,
+          "ratio": 9.22
+        },
+        {
+          "token": "link",
+          "share_a": 0.6,
+          "share_b": 0.074,
+          "ratio": 8.5
+        },
+        {
+          "token": "read",
+          "share_a": 0.6,
+          "share_b": 0.084,
+          "ratio": 7.53
+        },
+        {
+          "token": "life",
+          "share_a": 0.6,
+          "share_b": 0.099,
+          "ratio": 6.36
+        },
+        {
+          "token": "www",
+          "share_a": 0.8,
+          "share_b": 0.148,
+          "ratio": 5.5
+        },
+        {
+          "token": "http",
+          "share_a": 0.8,
+          "share_b": 0.193,
+          "ratio": 4.23
+        },
+        {
+          "token": "new",
+          "share_a": 0.6,
+          "share_b": 0.161,
+          "ratio": 3.92
+        },
+        {
+          "token": "make",
+          "share_a": 0.6,
+          "share_b": 0.161,
+          "ratio": 3.92
+        },
+        {
+          "token": "get",
+          "share_a": 0.6,
+          "share_b": 0.202,
+          "ratio": 3.13
+        },
+        {
+          "token": "com",
+          "share_a": 1.0,
+          "share_b": 0.337,
+          "ratio": 2.97
+        },
+        {
+          "token": "message",
+          "share_a": 0.6,
+          "share_b": 0.214,
+          "ratio": 2.96
+        },
+        {
+          "token": "e-mail",
+          "share_a": 0.6,
+          "share_b": 0.218,
+          "ratio": 2.91
+        },
+        {
+          "token": "receive",
+          "share_a": 0.6,
+          "share_b": 0.218,
+          "ratio": 2.91
+        },
+        {
+          "token": "only",
+          "share_a": 0.6,
+          "share_b": 0.235,
+          "ratio": 2.69
+        },
+        {
+          "token": "about",
+          "share_a": 0.6,
+          "share_b": 0.239,
+          "ratio": 2.65
+        }
+      ],
+      "spam_as_phish_tokens": []
+    }
+  },
+  "plots": {
+    "confusion_matrix": "reports/eval_ft-weighted/confusion_matrix_test3.png",
+    "confusion_matrix3": "reports/eval_ft-weighted/confusion_matrix3_test3.png",
+    "roc_curve": "reports/eval_ft-weighted/roc_curve_test3.png",
+    "pr_curve": "reports/eval_ft-weighted/pr_curve_test3.png"
+  }
+}
diff --git a/presentation/reports/eval_pretrain_curves.png b/presentation/reports/eval_pretrain_curves.png
new file mode 100644
index 0000000000000000000000000000000000000000..373f721ea3e201c9f5e7782d596011ff8474d806
Binary files /dev/null and b/presentation/reports/eval_pretrain_curves.png differ
diff --git a/presentation/reports/eval_pretrain_curves_test3.png b/presentation/reports/eval_pretrain_curves_test3.png
new file mode 100644
index 0000000000000000000000000000000000000000..373f721ea3e201c9f5e7782d596011ff8474d806
Binary files /dev/null and b/presentation/reports/eval_pretrain_curves_test3.png differ
diff --git a/presentation/reports/experiments.jsonl b/presentation/reports/experiments.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..fff2a2c09d0cc4fe0296c276f8d7f793c0b00d2b
--- /dev/null
+++ b/presentation/reports/experiments.jsonl
@@ -0,0 +1,8 @@
+{"run_name": "ft-ce", "timestamp": "2026-10-09T04:28:29", "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final", "dials": {"n_layers": 24, "frozen_layers": 0, "epochs_planned": 3, "epochs_run": 1.248, "loss": "cross_entropy", "focal_gamma": null, "lr": 3e-05, "batch_size": 32, "micro_batch": 16, "class_weights": null, "synthetic": "off", "synthetic_rows": 0, "warmup_fraction": 0.06, "max_steps": null, "limit_rows": null, "seed": 42, "schedule": "linear_decay", "selection_metric": "val_f1_unwanted", "pooling": "cls", "num_labels": 3, "device": "cuda"}, "params": {"total": 337444867, "trainable": 337444867, "embeddings": 34080768, "encoder_layers": 302309376, "final_norm": 2048, "head": 1052675}, "steps_run": 6250, "n_train": 160263, "n_val": 15284, "train_seconds": 3089.2, "peak_vram_gb": 13.133, "best_epoch": 0, "best_step": 5000, "best_score": 0.992930086410055, "val": {"accuracy": 0.9923449358806595, "precision": 0.9938308939155679, "recall": 0.992030910408114, "f1": 0.992930086410055, "roc_auc": 0.9995503578823617, "pr_auc": 0.9996107014519675, "f1_unwanted": 0.992930086410055, "macro_f1_3way": 0.9781799660145108, "per_class": {"Legitimate": {"precision": 0.9988504095416008, "recall": 0.9927163667523564, "f1": 0.995773941694721}, "Phishing": {"precision": 0.9390581717451524, "recall": 0.9970588235294118, "f1": 0.9671897289586305}, "Spam": {"precision": 0.9579617834394905, "recall": 0.9855832241153342, "f1": 0.9715762273901809}}, "three_way": {"n": 8105, "accuracy": 0.9922270203578039, "macro_f1": 0.9781799660145108, "per_class": {"Legitimate": {"precision": 0.9988504095416008, "recall": 0.9927163667523564, "f1": 0.995773941694721}, "Phishing": {"precision": 0.9390581717451524, "recall": 0.9970588235294118, "f1": 0.9671897289586305}, "Spam": {"precision": 0.9579617834394905, "recall": 0.9855832241153342, "f1": 0.9715762273901809}}, "confusion": {"order": ["Legitimate", "Phishing", "Spam"], "rows_true_cols_pred": [[6951, 18, 33], [1, 339, 0], [7, 4, 752]]}, "phish_vs_spam_on_true_unwanted": {"n": 1103, "order": ["Phishing", "Spam"], "rows_true_cols_pred": [[340, 0], [4, 759]]}}}, "val_threshold": 0.5, "label_names": ["Legitimate", "Phishing", "Spam"], "stopped_by": "early_stopping", "resumed_from_step": null, "test": null}
+{"run_name": "ft-weighted", "timestamp": "2026-10-09T05:55:09", "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final", "dials": {"n_layers": 24, "frozen_layers": 0, "epochs_planned": 2, "epochs_run": 2.0, "loss": "weighted_cross_entropy", "focal_gamma": null, "lr": 3e-05, "batch_size": 32, "micro_batch": 16, "class_weights": "inverse_frequency", "synthetic": "off", "synthetic_rows": 0, "warmup_fraction": 0.06, "max_steps": null, "limit_rows": null, "seed": 42, "schedule": "linear_decay", "selection_metric": "val_f1_unwanted", "pooling": "cls", "num_labels": 3, "device": "cuda"}, "params": {"total": 337444867, "trainable": 337444867, "embeddings": 34080768, "encoder_layers": 302309376, "final_norm": 2048, "head": 1052675}, "steps_run": 10018, "n_train": 160263, "n_val": 15284, "train_seconds": 5013.9, "peak_vram_gb": 13.133, "best_epoch": 1, "best_step": 10000, "best_score": 0.99373795761079, "val": {"accuracy": 0.9931954985605862, "precision": 0.9911121787172712, "recall": 0.9963776865491427, "f1": 0.99373795761079, "roc_auc": 0.9997532357160727, "pr_auc": 0.9997920087331971, "f1_unwanted": 0.99373795761079, "macro_f1_3way": 0.9774841428658321, "per_class": {"Legitimate": {"precision": 0.9992789154888952, "recall": 0.9895744073121965, "f1": 0.9944029850746269}, "Phishing": {"precision": 0.9630681818181818, "recall": 0.9970588235294118, "f1": 0.9797687861271677}, "Spam": {"precision": 0.9255189255189256, "recall": 0.9934469200524246, "f1": 0.9582806573957017}}, "three_way": {"n": 8105, "accuracy": 0.9902529302899444, "macro_f1": 0.9774841428658321, "per_class": {"Legitimate": {"precision": 0.9992789154888952, "recall": 0.9895744073121965, "f1": 0.9944029850746269}, "Phishing": {"precision": 0.9630681818181818, "recall": 0.9970588235294118, "f1": 0.9797687861271677}, "Spam": {"precision": 0.9255189255189256, "recall": 0.9934469200524246, "f1": 0.9582806573957017}}, "confusion": {"order": ["Legitimate", "Phishing", "Spam"], "rows_true_cols_pred": [[6929, 12, 61], [1, 339, 0], [4, 1, 758]]}, "phish_vs_spam_on_true_unwanted": {"n": 1103, "order": ["Phishing", "Spam"], "rows_true_cols_pred": [[339, 1], [2, 761]]}}}, "val_threshold": 0.5, "label_names": ["Legitimate", "Phishing", "Spam"], "stopped_by": "epochs_done", "resumed_from_step": null, "test": null}
+{"run_name": "ft-focal", "timestamp": "2026-10-09T07:21:55", "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final", "dials": {"n_layers": 24, "frozen_layers": 0, "epochs_planned": 2, "epochs_run": 2.0, "loss": "focal", "focal_gamma": 2.0, "lr": 3e-05, "batch_size": 32, "micro_batch": 16, "class_weights": null, "synthetic": "off", "synthetic_rows": 0, "warmup_fraction": 0.06, "max_steps": null, "limit_rows": null, "seed": 42, "schedule": "linear_decay", "selection_metric": "val_f1_unwanted", "pooling": "cls", "num_labels": 3, "device": "cuda"}, "params": {"total": 337444867, "trainable": 337444867, "embeddings": 34080768, "encoder_layers": 302309376, "final_norm": 2048, "head": 1052675}, "steps_run": 10018, "n_train": 160263, "n_val": 15284, "train_seconds": 5033.3, "peak_vram_gb": 13.133, "best_epoch": 1, "best_step": 8750, "best_score": 0.9951084002657166, "val": {"accuracy": 0.9947003402250719, "precision": 0.9954089646007007, "recall": 0.9948080173871046, "f1": 0.9951084002657166, "roc_auc": 0.9998186946414248, "pr_auc": 0.9998464119156345, "f1_unwanted": 0.9951084002657166, "macro_f1_3way": 0.9868751212108838, "per_class": {"Legitimate": {"precision": 0.9988533753762362, "recall": 0.99528706083976, "f1": 0.9970670291151013}, "Phishing": {"precision": 0.9853372434017595, "recall": 0.9882352941176471, "f1": 0.986784140969163}, "Spam": {"precision": 0.9618805590851334, "recall": 0.9921363040629095, "f1": 0.9767741935483871}}, "three_way": {"n": 8105, "accuracy": 0.994694632942628, "macro_f1": 0.9868751212108838, "per_class": {"Legitimate": {"precision": 0.9988533753762362, "recall": 0.99528706083976, "f1": 0.9970670291151013}, "Phishing": {"precision": 0.9853372434017595, "recall": 0.9882352941176471, "f1": 0.986784140969163}, "Spam": {"precision": 0.9618805590851334, "recall": 0.9921363040629095, "f1": 0.9767741935483871}}, "confusion": {"order": ["Legitimate", "Phishing", "Spam"], "rows_true_cols_pred": [[6969, 4, 29], [3, 336, 1], [5, 1, 757]]}, "phish_vs_spam_on_true_unwanted": {"n": 1103, "order": ["Phishing", "Spam"], "rows_true_cols_pred": [[339, 1], [1, 762]]}}}, "val_threshold": 0.5, "label_names": ["Legitimate", "Phishing", "Spam"], "stopped_by": "early_stopping", "resumed_from_step": null, "test": null}
+{"run_name": "ft-synthetic", "timestamp": "2026-10-09T08:16:37", "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final", "dials": {"n_layers": 24, "frozen_layers": 0, "epochs_planned": 2, "epochs_run": 1.232, "loss": "cross_entropy", "focal_gamma": null, "lr": 3e-05, "batch_size": 32, "micro_batch": 16, "class_weights": null, "synthetic": "on", "synthetic_rows": 2000, "warmup_fraction": 0.06, "max_steps": null, "limit_rows": null, "seed": 42, "schedule": "linear_decay", "selection_metric": "val_f1_unwanted", "pooling": "cls", "num_labels": 3, "device": "cuda"}, "params": {"total": 337444867, "trainable": 337444867, "embeddings": 34080768, "encoder_layers": 302309376, "final_norm": 2048, "head": 1052675}, "steps_run": 6250, "n_train": 162263, "n_val": 15284, "train_seconds": 3118.3, "peak_vram_gb": 13.131, "best_epoch": 0, "best_step": 5000, "best_score": 0.9926514877725575, "val": {"accuracy": 0.9920177963883801, "precision": 0.9903846153846154, "recall": 0.9949287611687998, "f1": 0.9926514877725575, "roc_auc": 0.9996055651398734, "pr_auc": 0.9996662445428656, "f1_unwanted": 0.9926514877725575, "macro_f1_3way": 0.9665378162175431, "per_class": {"Legitimate": {"precision": 0.9992783951508154, "recall": 0.988860325621251, "f1": 0.9940420644605555}, "Phishing": {"precision": 0.8805194805194805, "recall": 0.9970588235294118, "f1": 0.9351724137931035}, "Spam": {"precision": 0.9532237673830595, "recall": 0.9882044560943644, "f1": 0.9703989703989704}}, "three_way": {"n": 8105, "accuracy": 0.9891425046267736, "macro_f1": 0.9665378162175431, "per_class": {"Legitimate": {"precision": 0.9992783951508154, "recall": 0.988860325621251, "f1": 0.9940420644605555}, "Phishing": {"precision": 0.8805194805194805, "recall": 0.9970588235294118, "f1": 0.9351724137931035}, "Spam": {"precision": 0.9532237673830595, "recall": 0.9882044560943644, "f1": 0.9703989703989704}}, "confusion": {"order": ["Legitimate", "Phishing", "Spam"], "rows_true_cols_pred": [[6924, 41, 37], [1, 339, 0], [4, 5, 754]]}, "phish_vs_spam_on_true_unwanted": {"n": 1103, "order": ["Phishing", "Spam"], "rows_true_cols_pred": [[340, 0], [6, 757]]}}}, "val_threshold": 0.5, "label_names": ["Legitimate", "Phishing", "Spam"], "stopped_by": "early_stopping", "resumed_from_step": null, "test": null}
+{"run_name": "ft-12L", "timestamp": "2026-10-09T08:54:49", "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final", "dials": {"n_layers": 12, "frozen_layers": 0, "epochs_planned": 2, "epochs_run": 1.747, "loss": "cross_entropy", "focal_gamma": null, "lr": 3e-05, "batch_size": 32, "micro_batch": 16, "class_weights": null, "synthetic": "off", "synthetic_rows": 0, "warmup_fraction": 0.06, "max_steps": null, "limit_rows": null, "seed": 42, "schedule": "linear_decay", "selection_metric": "val_f1_unwanted", "pooling": "cls", "num_labels": 3, "device": "cuda"}, "params": {"total": 186290179, "trainable": 186290179, "embeddings": 34080768, "encoder_layers": 151154688, "final_norm": 2048, "head": 1052675}, "steps_run": 8750, "n_train": 160263, "n_val": 15284, "train_seconds": 2118.2, "peak_vram_gb": 6.969, "best_epoch": 1, "best_step": 6250, "best_score": 0.9925057415689592, "val": {"accuracy": 0.9918869405914682, "precision": 0.9935866408518877, "recall": 0.9914271914996378, "f1": 0.9925057415689592, "roc_auc": 0.9995007118744352, "pr_auc": 0.9995737781814353, "f1_unwanted": 0.9925057415689592, "macro_f1_3way": 0.9774079655816474, "per_class": {"Legitimate": {"precision": 0.9984195402298851, "recall": 0.9924307340759783, "f1": 0.9954161294943418}, "Phishing": {"precision": 0.9570200573065902, "recall": 0.9823529411764705, "f1": 0.969521044992743}, "Spam": {"precision": 0.9472361809045227, "recall": 0.9882044560943644, "f1": 0.9672867222578576}}, "three_way": {"n": 8105, "accuracy": 0.9916101172115978, "macro_f1": 0.9774079655816474, "per_class": {"Legitimate": {"precision": 0.9984195402298851, "recall": 0.9924307340759783, "f1": 0.9954161294943418}, "Phishing": {"precision": 0.9570200573065902, "recall": 0.9823529411764705, "f1": 0.969521044992743}, "Spam": {"precision": 0.9472361809045227, "recall": 0.9882044560943644, "f1": 0.9672867222578576}}, "confusion": {"order": ["Legitimate", "Phishing", "Spam"], "rows_true_cols_pred": [[6949, 13, 40], [4, 334, 2], [7, 2, 754]]}, "phish_vs_spam_on_true_unwanted": {"n": 1103, "order": ["Phishing", "Spam"], "rows_true_cols_pred": [[338, 2], [2, 761]]}}}, "val_threshold": 0.5, "label_names": ["Legitimate", "Phishing", "Spam"], "stopped_by": "early_stopping", "resumed_from_step": null, "test": null}
+{"run_name": "ft-scratch", "timestamp": "2026-10-09T09:57:34", "init": "scratch", "dials": {"n_layers": 24, "frozen_layers": 0, "epochs_planned": 2, "epochs_run": 1.497, "loss": "cross_entropy", "focal_gamma": null, "lr": 3e-05, "batch_size": 32, "micro_batch": 16, "class_weights": null, "synthetic": "off", "synthetic_rows": 0, "warmup_fraction": 0.06, "max_steps": null, "limit_rows": null, "seed": 42, "schedule": "linear_decay", "selection_metric": "val_f1_unwanted", "pooling": "cls", "num_labels": 3, "device": "cuda"}, "params": {"total": 337444867, "trainable": 337444867, "embeddings": 34080768, "encoder_layers": 302309376, "final_norm": 2048, "head": 1052675}, "steps_run": 7500, "n_train": 160263, "n_val": 15284, "train_seconds": 3656.4, "peak_vram_gb": 13.133, "best_epoch": 0, "best_step": 5009, "best_score": 0.9874562778916898, "val": {"accuracy": 0.9863909971211725, "precision": 0.9863855421686747, "recall": 0.9885293407389519, "f1": 0.9874562778916898, "roc_auc": 0.9984844430897414, "pr_auc": 0.998618660839301, "f1_unwanted": 0.9874562778916898, "macro_f1_3way": 0.9537815709229355, "per_class": {"Legitimate": {"precision": 0.9979745370370371, "recall": 0.9851471008283348, "f1": 0.991519333045853}, "Phishing": {"precision": 0.9384164222873901, "recall": 0.9411764705882353, "f1": 0.9397944199706314}, "Spam": {"precision": 0.8814553990610329, "recall": 0.9842726081258192, "f1": 0.930030959752322}}, "three_way": {"n": 8105, "accuracy": 0.9832202344231955, "macro_f1": 0.9537815709229355, "per_class": {"Legitimate": {"precision": 0.9979745370370371, "recall": 0.9851471008283348, "f1": 0.991519333045853}, "Phishing": {"precision": 0.9384164222873901, "recall": 0.9411764705882353, "f1": 0.9397944199706314}, "Spam": {"precision": 0.8814553990610329, "recall": 0.9842726081258192, "f1": 0.930030959752322}}, "confusion": {"order": ["Legitimate", "Phishing", "Spam"], "rows_true_cols_pred": [[6898, 20, 84], [3, 320, 17], [11, 1, 751]]}, "phish_vs_spam_on_true_unwanted": {"n": 1103, "order": ["Phishing", "Spam"], "rows_true_cols_pred": [[322, 18], [1, 762]]}}}, "val_threshold": 0.5, "label_names": ["Legitimate", "Phishing", "Spam"], "stopped_by": "early_stopping", "resumed_from_step": null, "test": null}
+{"run_name": "ft-ce", "timestamp": "2026-10-09 09:59", "stage": "evaluate", "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-ce/best", "splits": ["test", "test3"], "num_labels": 3, "dev_run": false, "limit_rows": null, "val": null, "test": {"accuracy": 0.9495153968838179, "precision": 0.8610392664001881, "recall": 0.9404211607601438, "f1": 0.8989812200810113, "specificity": 0.9523694390715667, "roc_auc": 0.9874951859346837, "pr_auc": 0.9625515458938435, "recall_at_fpr_1pct": 0.7745249101181304, "recall_at_fpr_0_1pct": 0.35285053929121724, "confusion_matrix": {"tn": 11817, "fp": 591, "fn": 232, "tp": 3662}, "n": 16302, "threshold": 0.5, "lure_recall": 0.9958847736625515, "f1_unwanted": 0.8989812200810113, "threeway": null}, "test3": {"accuracy": 0.989751552795031, "precision": 0.9876847290640394, "recall": 0.9852579852579852, "f1": 0.986469864698647, "specificity": 0.992496248124062, "roc_auc": 0.9991504761389705, "pr_auc": 0.9985052105989761, "recall_at_fpr_1pct": 0.9868959868959869, "recall_at_fpr_0_1pct": 0.868959868959869, "confusion_matrix": {"tn": 1984, "fp": 15, "fn": 18, "tp": 1203}, "n": 3220, "threshold": 0.5, "lure_recall": 0.9925373134328358, "f1_unwanted": 0.986469864698647, "threeway": {"n": 3220, "accuracy": 0.986335403726708, "macro_f1": 0.9821678741985428, "macro_f1_present_classes": 0.9821678741985428, "per_class": {"Legitimate": {"precision": 0.991008991008991, "recall": 0.992496248124062, "f1": 0.9917520619845038, "support": 1999}, "Phishing": {"precision": 0.9806576402321083, "recall": 0.9731285988483686, "f1": 0.976878612716763, "support": 521}, "Spam": {"precision": 0.9771754636233951, "recall": 0.9785714285714285, "f1": 0.9778729478943612, "support": 700}}, "confusion_matrix_3x3": [[1984, 4, 11], [9, 507, 5], [9, 6, 685]], "kind_accuracy_on_true_unwanted": 0.9893529893529893}, "macro_f1_3way": 0.9821678741985428}, "latency_ms": {"gpu_bf16_avg": 9.89, "cpu_fp32_avg": 513.57}}
+{"run_name": "ft-focal", "timestamp": "2026-10-09 10:46", "stage": "evaluate", "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best", "splits": ["test", "test3"], "num_labels": 3, "dev_run": false, "limit_rows": null, "val": null, "test": {"accuracy": 0.9619065145380935, "precision": 0.8794342684906098, "recall": 0.9740626605033385, "f1": 0.9243328865602534, "specificity": 0.9580915538362347, "roc_auc": 0.9946343971962354, "pr_auc": 0.9847899987683836, "recall_at_fpr_1pct": 0.8954802259887006, "recall_at_fpr_0_1pct": 0.6497175141242938, "confusion_matrix": {"tn": 11888, "fp": 520, "fn": 101, "tp": 3793}, "n": 16302, "threshold": 0.5, "lure_recall": 0.9958847736625515, "f1_unwanted": 0.9243328865602534, "threeway": null}, "test3": {"accuracy": 0.989751552795031, "precision": 0.9892915980230642, "recall": 0.9836199836199836, "f1": 0.986447638603696, "specificity": 0.9934967483741871, "roc_auc": 0.9993325901279878, "pr_auc": 0.9988537821967886, "recall_at_fpr_1pct": 0.9868959868959869, "recall_at_fpr_0_1pct": 0.8787878787878788, "confusion_matrix": {"tn": 1986, "fp": 13, "fn": 20, "tp": 1201}, "n": 3220, "threshold": 0.5, "lure_recall": 1.0, "f1_unwanted": 0.986447638603696, "threeway": {"n": 3220, "accuracy": 0.9885093167701864, "macro_f1": 0.9867305252015051, "macro_f1_present_classes": 0.9867305252015051, "per_class": {"Legitimate": {"precision": 0.9900299102691924, "recall": 0.9934967483741871, "f1": 0.9917602996254682, "support": 1999}, "Phishing": {"precision": 0.9961240310077519, "recall": 0.9865642994241842, "f1": 0.991321118611379, "support": 521}, "Spam": {"precision": 0.9785100286532952, "recall": 0.9757142857142858, "f1": 0.977110157367668, "support": 700}}, "confusion_matrix_3x3": [[1986, 1, 12], [4, 514, 3], [16, 1, 683]], "kind_accuracy_on_true_unwanted": 0.9959049959049959}, "macro_f1_3way": 0.9867305252015051}, "latency_ms": {"gpu_bf16_avg": 10.08, "cpu_fp32_avg": 458.57}}
diff --git a/presentation/reports/finetune_curves.png b/presentation/reports/finetune_curves.png
new file mode 100644
index 0000000000000000000000000000000000000000..ae1bc0c23e860d8909c3bf703c397daa8f0ef507
--- /dev/null
+++ b/presentation/reports/finetune_curves.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fc59e978493e02abd559bb4247ca506fbca7feee209756f6e67fba40edf7895
+size 104005
diff --git a/presentation/reports/finetune_curves_ft-12L.png b/presentation/reports/finetune_curves_ft-12L.png
new file mode 100644
index 0000000000000000000000000000000000000000..795bd45f7f5f859c6be3fa91ff591aed617abdb8
--- /dev/null
+++ b/presentation/reports/finetune_curves_ft-12L.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a727e1e77f8116cc685e5b8620a1b4d2c2bd00afaabcb7504bff7608ae2656c
+size 103837
diff --git a/presentation/reports/finetune_curves_ft-ce.png b/presentation/reports/finetune_curves_ft-ce.png
new file mode 100644
index 0000000000000000000000000000000000000000..67ed78111ef5e0e7a3e3865c9b0e9a0d2f1e2b4a
--- /dev/null
+++ b/presentation/reports/finetune_curves_ft-ce.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f0912d526157afa013354d593f74b4bc072e5fcdfe64d42952986c068ad996f0
+size 101762
diff --git a/presentation/reports/finetune_curves_ft-focal.png b/presentation/reports/finetune_curves_ft-focal.png
new file mode 100644
index 0000000000000000000000000000000000000000..ae1bc0c23e860d8909c3bf703c397daa8f0ef507
--- /dev/null
+++ b/presentation/reports/finetune_curves_ft-focal.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fc59e978493e02abd559bb4247ca506fbca7feee209756f6e67fba40edf7895
+size 104005
diff --git a/presentation/reports/finetune_curves_ft-scratch.png b/presentation/reports/finetune_curves_ft-scratch.png
new file mode 100644
index 0000000000000000000000000000000000000000..5221f82eaff6a64dd32290acee2afc6501165a59
--- /dev/null
+++ b/presentation/reports/finetune_curves_ft-scratch.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf6d870b9240adaca9ac122907f590061af7b31e8c1f9f3f2155cf14cdf6773e
+size 116255
diff --git a/presentation/reports/finetune_curves_ft-synthetic.png b/presentation/reports/finetune_curves_ft-synthetic.png
new file mode 100644
index 0000000000000000000000000000000000000000..b5d3b8ea09f321b27c823cf02128b0f8206fc7be
--- /dev/null
+++ b/presentation/reports/finetune_curves_ft-synthetic.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8530480ad1b77e6a4729ebde320fa959829b7801aad8fdfbcb5c52ebfbf3aa26
+size 108054
diff --git a/presentation/reports/finetune_curves_ft-weighted.png b/presentation/reports/finetune_curves_ft-weighted.png
new file mode 100644
index 0000000000000000000000000000000000000000..90b0645b47582c375b35dfb687db299df54f5006
--- /dev/null
+++ b/presentation/reports/finetune_curves_ft-weighted.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90e75f1a7d514d841b8617e9699cf3b58f6cf70f73b9b6cece750ebf6c6310d0
+size 108478
diff --git a/presentation/reports/latency.json b/presentation/reports/latency.json
new file mode 100644
index 0000000000000000000000000000000000000000..3934fa4aed2e0311fb465ef9ca1b1b5809c33811
--- /dev/null
+++ b/presentation/reports/latency.json
@@ -0,0 +1,38 @@
+{
+  "instructor_benchmark_ms": 74.0,
+  "samples_requested": 100,
+  "sample_split": "test",
+  "max_len": 512,
+  "gpu_bf16": {
+    "n": 100,
+    "avg_ms": 10.08,
+    "p50_ms": 8.9,
+    "p95_ms": 13.72,
+    "max_ms": 18.58,
+    "avg_s": 0.0101,
+    "mean_tokens": 280.0,
+    "ratio_to_benchmark": 0.136,
+    "device": "NVIDIA GeForce RTX 3090 Ti"
+  },
+  "cpu_fp32": {
+    "n": 100,
+    "avg_ms": 458.57,
+    "p50_ms": 445.01,
+    "p95_ms": 701.07,
+    "max_ms": 828.65,
+    "avg_s": 0.4586,
+    "mean_tokens": 280.0,
+    "ratio_to_benchmark": 6.197,
+    "threads": 16,
+    "estimated_full_run_s": 41.8
+  },
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+  "run_name": "ft-focal",
+  "params": 337444867,
+  "timestamp": "2026-10-09 10:46",
+  "num_labels": 3,
+  "measured_once_for_splits": [
+    "test",
+    "test3"
+  ]
+}
diff --git a/presentation/reports/metrics.json b/presentation/reports/metrics.json
new file mode 100644
index 0000000000000000000000000000000000000000..b22a6297aa3dddbbc0785550dc95df4697ef13f9
--- /dev/null
+++ b/presentation/reports/metrics.json
@@ -0,0 +1,1111 @@
+{
+  "timestamp": "2026-10-09 10:46",
+  "run_name": "ft-focal",
+  "split": "test",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout.parquet",
+  "split_title": "TREC-06 binary holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-focal",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "focal",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 1,
+    "best_step": 8750,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9951084002657166,
+    "val_metrics": {
+      "accuracy": 0.9947003402250719,
+      "precision": 0.9954089646007007,
+      "recall": 0.9948080173871046,
+      "f1": 0.9951084002657166,
+      "roc_auc": 0.9998186946414248,
+      "pr_auc": 0.9998464119156345,
+      "tn": 6964,
+      "fp": 38,
+      "fn": 43,
+      "tp": 8239,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.42074909806251526,
+      "f1_at_optimal_threshold": 0.9952995058454863,
+      "f1_unwanted": 0.9951084002657166,
+      "n_3way": 8105,
+      "accuracy_3way": 0.994694632942628,
+      "macro_f1_3way": 0.9868751212108838,
+      "precision_legitimate": 0.9988533753762362,
+      "recall_legitimate": 0.99528706083976,
+      "f1_legitimate": 0.9970670291151013,
+      "precision_phishing": 0.9853372434017595,
+      "recall_phishing": 0.9882352941176471,
+      "f1_phishing": 0.986784140969163,
+      "precision_spam": 0.9618805590851334,
+      "recall_spam": 0.9921363040629095,
+      "f1_spam": 0.9767741935483871,
+      "cm3_ll": 6969,
+      "cm3_lp": 4,
+      "cm3_ls": 29,
+      "cm3_pl": 3,
+      "cm3_pp": 336,
+      "cm3_ps": 1,
+      "cm3_sl": 5,
+      "cm3_sp": 1,
+      "cm3_ss": 757,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 339,
+      "pvs_phish_as_spam": 1,
+      "pvs_spam_as_phish": 1,
+      "pvs_spam_as_spam": 762,
+      "val_loss": 0.030296484990695233,
+      "n": 15284,
+      "eval_seconds": 48.0
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9947003402250719,
+        "precision": 0.9954089646007007,
+        "recall": 0.9948080173871046,
+        "f1": 0.9951084002657166,
+        "roc_auc": 0.9998186946414248,
+        "pr_auc": 0.9998464119156345,
+        "tn": 6964,
+        "fp": 38,
+        "fn": 43,
+        "tp": 8239,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.42074909806251526,
+        "f1_at_optimal_threshold": 0.9952995058454863,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.994694632942628,
+        "macro_f1": 0.9868751212108838,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9988533753762362,
+            "recall": 0.99528706083976,
+            "f1": 0.9970670291151013
+          },
+          "Phishing": {
+            "precision": 0.9853372434017595,
+            "recall": 0.9882352941176471,
+            "f1": 0.986784140969163
+          },
+          "Spam": {
+            "precision": 0.9618805590851334,
+            "recall": 0.9921363040629095,
+            "f1": 0.9767741935483871
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6969,
+              4,
+              29
+            ],
+            [
+              3,
+              336,
+              1
+            ],
+            [
+              5,
+              1,
+              757
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              339,
+              1
+            ],
+            [
+              1,
+              762
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.42074909806251526,
+      "val_f1_at_optimal": 0.9952995058454863
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 16302,
+    "positive": 3894,
+    "negative": 12408,
+    "positive_rate": 0.2389,
+    "label3": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "by_source": {
+      "TREC-06": {
+        "rows": 16302,
+        "positive": 3894,
+        "label3": {
+          "legit": 12408,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 3894
+        }
+      }
+    },
+    "mean_tokens_full": 710.6,
+    "pct_truncated": 21.93
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.9619065145380935,
+    "precision": 0.8794342684906098,
+    "recall": 0.9740626605033385,
+    "f1": 0.9243328865602534,
+    "specificity": 0.9580915538362347,
+    "false_positive_rate": 0.041908446163765316,
+    "false_negative_rate": 0.02593733949666153,
+    "confusion_matrix": {
+      "tn": 11888,
+      "fp": 520,
+      "fn": 101,
+      "tp": 3793
+    },
+    "n": 16302,
+    "n_positive": 3894,
+    "n_negative": 12408
+  },
+  "threshold_free": {
+    "roc_auc": 0.9946343971962354,
+    "pr_auc": 0.9847899987683836,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.8954802259887006,
+      "fpr": 0.009993552546744036,
+      "threshold": 0.8660591840744019,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.6497175141242938,
+      "fpr": 0.0009671179883945841,
+      "threshold": 0.9728788733482361,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.6815109252929688,
+      "f1": 0.9397284608552213,
+      "precision": 0.9287684976172561,
+      "recall": 0.9509501797637391
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 12408,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 3894
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 3793,
+        "predicted_phishing": 974,
+        "predicted_spam": 2819
+      },
+      "false_positives_flagged": {
+        "n": 520,
+        "predicted_phishing": 175,
+        "predicted_spam": 345
+      },
+      "missed_positives": {
+        "n": 101,
+        "predicted_phishing": 35,
+        "predicted_spam": 66
+      },
+      "all_flagged": {
+        "n": 4313,
+        "predicted_phishing": 1149,
+        "predicted_spam": 3164
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 3793,
+        "predicted_phishing": 974,
+        "predicted_spam": 2819
+      }
+    },
+    "rows_with_full_label": 12408,
+    "rows_unwanted_only": 3894,
+    "evaluated": false,
+    "reason": "no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)"
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 129,
+      "bank account": 78,
+      "security alert": 70,
+      "within 24 hours": 65,
+      "urgently": 51,
+      "paypal account": 20,
+      "within 48 hours": 7,
+      "verify your account": 7,
+      "update your account": 6,
+      "account will be suspended": 5,
+      "ebay account": 5,
+      "enter your password": 5,
+      "wire transfer": 5,
+      "unusual activity": 5,
+      "confirm your identity": 4,
+      "verify your identity": 4,
+      "click here to verify": 3,
+      "log in to your account": 2,
+      "confirm your account": 2,
+      "security notice": 1,
+      "gift card": 1,
+      "reset your password": 1,
+      "account has been locked": 1,
+      "update your information": 1,
+      "account has been limited": 1,
+      "login to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 5,
+        "negative": 0
+      },
+      "update your account": {
+        "positive": 6,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 0,
+        "negative": 1
+      },
+      "click here to verify": {
+        "positive": 3,
+        "negative": 0
+      },
+      "account has been limited": {
+        "positive": 1,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 4,
+        "negative": 1
+      },
+      "account has been locked": {
+        "positive": 1,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 2,
+        "negative": 3
+      },
+      "paypal account": {
+        "positive": 20,
+        "negative": 0
+      },
+      "ebay account": {
+        "positive": 5,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "wire transfer": {
+        "positive": 5,
+        "negative": 0
+      },
+      "gift card": {
+        "positive": 0,
+        "negative": 1
+      },
+      "urgent": {
+        "positive": 81,
+        "negative": 48
+      },
+      "urgently": {
+        "positive": 22,
+        "negative": 29
+      },
+      "security alert": {
+        "positive": 2,
+        "negative": 68
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "login to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 2,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 71,
+        "negative": 7
+      },
+      "within 24 hours": {
+        "positive": 63,
+        "negative": 2
+      },
+      "within 48 hours": {
+        "positive": 4,
+        "negative": 3
+      }
+    },
+    "label_purity": 0.6106,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 155,
+      "phish": 0,
+      "spam": 0,
+      "unwanted": 243
+    },
+    "threeway_accuracy": null,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 0.9824120603015075,
+      "precision": 0.9758064516129032,
+      "recall": 0.9958847736625515,
+      "f1": 0.9857433808553973,
+      "specificity": 0.9612903225806452,
+      "false_positive_rate": 0.03870967741935484,
+      "false_negative_rate": 0.00411522633744856,
+      "confusion_matrix": {
+        "tn": 149,
+        "fp": 6,
+        "fn": 1,
+        "tp": 242
+      },
+      "n": 398,
+      "n_positive": 243,
+      "n_negative": 155,
+      "roc_auc": 0.9984866587017125
+    }
+  },
+  "per_source": {
+    "TREC-06": {
+      "threshold": 0.5,
+      "accuracy": 0.9619065145380935,
+      "precision": 0.8794342684906098,
+      "recall": 0.9740626605033385,
+      "f1": 0.9243328865602534,
+      "specificity": 0.9580915538362347,
+      "false_positive_rate": 0.041908446163765316,
+      "false_negative_rate": 0.02593733949666153,
+      "confusion_matrix": {
+        "tn": 11888,
+        "fp": 520,
+        "fn": 101,
+        "tp": 3793
+      },
+      "n": 16302,
+      "n_positive": 3894,
+      "n_negative": 12408,
+      "roc_auc": 0.9946343971962354,
+      "label3_counts": {
+        "legit": 12408,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 3894
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 3793,
+        "mean_tokens_full": 283.9,
+        "mean_tokens_fed": 232.2,
+        "median_tokens_full": 207.0,
+        "pct_truncated": 15.3
+      },
+      "tn": {
+        "n": 11888,
+        "mean_tokens_full": 827.0,
+        "mean_tokens_fed": 294.4,
+        "median_tokens_full": 269.0,
+        "pct_truncated": 23.8
+      },
+      "fp": {
+        "n": 520,
+        "mean_tokens_full": 1237.2,
+        "mean_tokens_fed": 276.6,
+        "median_tokens_full": 241.5,
+        "pct_truncated": 29.0
+      },
+      "fn": {
+        "n": 101,
+        "mean_tokens_full": 321.7,
+        "mean_tokens_fed": 220.4,
+        "median_tokens_full": 181.0,
+        "pct_truncated": 12.9
+      },
+      "correct": {
+        "n": 15681,
+        "mean_tokens_full": 695.6,
+        "mean_tokens_fed": 279.4,
+        "median_tokens_full": 247.0,
+        "pct_truncated": 21.8
+      },
+      "wrong": {
+        "n": 621,
+        "mean_tokens_full": 1088.3,
+        "mean_tokens_fed": 267.5,
+        "median_tokens_full": 213.0,
+        "pct_truncated": 26.4
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 12408,
+        "mean_tokens_full": 844.2,
+        "mean_tokens_fed": 293.7,
+        "median_tokens_full": 268.0,
+        "pct_truncated": 24.0
+      },
+      "unwanted": {
+        "n": 3894,
+        "mean_tokens_full": 284.9,
+        "mean_tokens_fed": 231.9,
+        "median_tokens_full": 206.0,
+        "pct_truncated": 15.3
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 1363,
+        "n_positive": 587,
+        "positive_rate": 0.43066764490095377,
+        "recall": 0.9744463373083475,
+        "fpr": 0.10695876288659793,
+        "accuracy": 0.9280997798972854
+      },
+      {
+        "bucket": "65-128",
+        "n": 2538,
+        "n_positive": 736,
+        "positive_rate": 0.2899921197793538,
+        "recall": 0.9619565217391305,
+        "fpr": 0.039400665926748055,
+        "accuracy": 0.9609929078014184
+      },
+      {
+        "bucket": "129-256",
+        "n": 4594,
+        "n_positive": 1196,
+        "positive_rate": 0.260339573356552,
+        "recall": 0.9765886287625418,
+        "fpr": 0.034137728075338436,
+        "accuracy": 0.9686547670875054
+      },
+      {
+        "bucket": "257-512",
+        "n": 4232,
+        "n_positive": 780,
+        "positive_rate": 0.1843100189035917,
+        "recall": 0.9782051282051282,
+        "fpr": 0.028679026651216685,
+        "accuracy": 0.9725897920604915
+      },
+      {
+        "bucket": ">512",
+        "n": 3575,
+        "n_positive": 595,
+        "positive_rate": 0.16643356643356644,
+        "recall": 0.9781512605042016,
+        "fpr": 0.05067114093959731,
+        "accuracy": 0.9541258741258741
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 16302,
+    "seconds": 57.39,
+    "rows_per_s": 284.1,
+    "padded_tokens": 4606544,
+    "real_tokens": 4546874,
+    "padded_tokens_per_s": 80267,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 10.08,
+      "p50_ms": 8.9,
+      "p95_ms": 13.72,
+      "max_ms": 18.58,
+      "avg_s": 0.0101,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.136,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 458.57,
+      "p50_ms": 445.01,
+      "p95_ms": 701.07,
+      "max_ms": 828.65,
+      "avg_s": 0.4586,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 6.197,
+      "threads": 16,
+      "estimated_full_run_s": 41.8
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+    "run_name": "ft-focal",
+    "params": 337444867,
+    "timestamp": "2026-10-09 10:46",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "/home/nathank/BITS/dev/Phishing Model/reports/error_analysis.md",
+    "n_fn": 101,
+    "n_fp": 520,
+    "composition": {
+      "fp": {
+        "n": 520,
+        "mailing_list_or_digest": 0.196,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.023,
+        "truncated_gt512": 0.29,
+        "lure_phrase": 0.012
+      },
+      "tn": {
+        "n": 11888,
+        "mailing_list_or_digest": 0.297,
+        "html_heavy": 0.002,
+        "very_short_lt20_tokens": 0.004,
+        "truncated_gt512": 0.238,
+        "lure_phrase": 0.013
+      },
+      "fn": {
+        "n": 101,
+        "mailing_list_or_digest": 0.089,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.129,
+        "lure_phrase": 0.01
+      },
+      "tp": {
+        "n": 3793,
+        "mailing_list_or_digest": 0.028,
+        "html_heavy": 0.001,
+        "very_short_lt20_tokens": 0.002,
+        "truncated_gt512": 0.153,
+        "lure_phrase": 0.064
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "smailex",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 934.36
+      },
+      {
+        "token": "blocki",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 934.36
+      },
+      {
+        "token": "scanmail",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 934.36
+      },
+      {
+        "token": "progra",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 934.36
+      },
+      {
+        "token": "scanning",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 934.36
+      },
+      {
+        "token": "matched",
+        "share_a": 0.119,
+        "share_b": 0.0,
+        "ratio": 311.45
+      },
+      {
+        "token": "tex-fonts",
+        "share_a": 0.079,
+        "share_b": 0.0,
+        "ratio": 211.79
+      },
+      {
+        "token": "pattern",
+        "share_a": 0.129,
+        "share_b": 0.001,
+        "ratio": 201.82
+      },
+      {
+        "token": "blocking",
+        "share_a": 0.129,
+        "share_b": 0.001,
+        "ratio": 201.82
+      },
+      {
+        "token": "tpn",
+        "share_a": 0.069,
+        "share_b": 0.001,
+        "ratio": 112.12
+      },
+      {
+        "token": "setting",
+        "share_a": 0.129,
+        "share_b": 0.001,
+        "ratio": 91.74
+      },
+      {
+        "token": "pif",
+        "share_a": 0.079,
+        "share_b": 0.001,
+        "ratio": 70.6
+      },
+      {
+        "token": "detected",
+        "share_a": 0.119,
+        "share_b": 0.002,
+        "ratio": 62.29
+      },
+      {
+        "token": "utah",
+        "share_a": 0.119,
+        "share_b": 0.002,
+        "ratio": 62.29
+      },
+      {
+        "token": "attachment",
+        "share_a": 0.149,
+        "share_b": 0.003,
+        "ratio": 50.37
+      },
+      {
+        "token": "math",
+        "share_a": 0.129,
+        "share_b": 0.003,
+        "ratio": 48.05
+      },
+      {
+        "token": "enroll",
+        "share_a": 0.05,
+        "share_b": 0.001,
+        "ratio": 45.68
+      },
+      {
+        "token": "seismic",
+        "share_a": 0.05,
+        "share_b": 0.001,
+        "ratio": 45.68
+      },
+      {
+        "token": "settings",
+        "share_a": 0.129,
+        "share_b": 0.003,
+        "ratio": 37.37
+      },
+      {
+        "token": "moved",
+        "share_a": 0.119,
+        "share_b": 0.003,
+        "ratio": 37.37
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "irc-list-web",
+        "share_a": 0.054,
+        "share_b": 0.0,
+        "ratio": 1301.91
+      },
+      {
+        "token": "quarantine",
+        "share_a": 0.054,
+        "share_b": 0.0,
+        "ratio": 118.36
+      },
+      {
+        "token": "foundstone",
+        "share_a": 0.05,
+        "share_b": 0.001,
+        "ratio": 80.7
+      },
+      {
+        "token": "skidmore",
+        "share_a": 0.054,
+        "share_b": 0.001,
+        "ratio": 76.58
+      },
+      {
+        "token": "monty",
+        "share_a": 0.052,
+        "share_b": 0.001,
+        "ratio": 66.12
+      },
+      {
+        "token": "bn3",
+        "share_a": 0.06,
+        "share_b": 0.002,
+        "ratio": 27.15
+      },
+      {
+        "token": "ips",
+        "share_a": 0.054,
+        "share_b": 0.002,
+        "ratio": 26.57
+      },
+      {
+        "token": "anyplace",
+        "share_a": 0.06,
+        "share_b": 0.002,
+        "ratio": 26.16
+      },
+      {
+        "token": "noc's",
+        "share_a": 0.054,
+        "share_b": 0.002,
+        "ratio": 23.67
+      },
+      {
+        "token": "https",
+        "share_a": 0.108,
+        "share_b": 0.006,
+        "ratio": 19.12
+      },
+      {
+        "token": "cgi",
+        "share_a": 0.06,
+        "share_b": 0.004,
+        "ratio": 16.17
+      },
+      {
+        "token": "procexp",
+        "share_a": 0.054,
+        "share_b": 0.003,
+        "ratio": 15.69
+      },
+      {
+        "token": "tcpview",
+        "share_a": 0.054,
+        "share_b": 0.003,
+        "ratio": 15.69
+      },
+      {
+        "token": "sysinternals",
+        "share_a": 0.054,
+        "share_b": 0.004,
+        "ratio": 15.32
+      },
+      {
+        "token": "filtered",
+        "share_a": 0.052,
+        "share_b": 0.003,
+        "ratio": 15.14
+      },
+      {
+        "token": "atkins",
+        "share_a": 0.054,
+        "share_b": 0.004,
+        "ratio": 14.63
+      },
+      {
+        "token": "rogue",
+        "share_a": 0.056,
+        "share_b": 0.004,
+        "ratio": 14.49
+      },
+      {
+        "token": "firewall",
+        "share_a": 0.071,
+        "share_b": 0.005,
+        "ratio": 13.93
+      },
+      {
+        "token": "inbox",
+        "share_a": 0.058,
+        "share_b": 0.004,
+        "ratio": 13.79
+      },
+      {
+        "token": "manage",
+        "share_a": 0.079,
+        "share_b": 0.006,
+        "ratio": 13.64
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:052869",
+      "primary:080991",
+      "primary:086987",
+      "primary:056767",
+      "primary:121268",
+      "primary:173636",
+      "primary:020947",
+      "primary:072993",
+      "primary:044992",
+      "primary:024565",
+      "primary:076023",
+      "primary:129745",
+      "primary:071577",
+      "primary:173938",
+      "primary:030013",
+      "primary:063077",
+      "primary:145723",
+      "primary:154489",
+      "primary:186523",
+      "primary:147887",
+      "primary:033014",
+      "primary:156328",
+      "primary:143065",
+      "primary:147193",
+      "primary:101618"
+    ],
+    "top_fp_doc_ids": [
+      "primary:015304",
+      "primary:040048",
+      "primary:194751",
+      "primary:072489",
+      "primary:194063",
+      "primary:033174",
+      "primary:101560",
+      "primary:159590",
+      "primary:184838",
+      "primary:011014",
+      "primary:193401",
+      "primary:072275",
+      "primary:170605",
+      "primary:175533",
+      "primary:131182",
+      "primary:106445",
+      "primary:078484",
+      "primary:196901",
+      "primary:143547",
+      "primary:129925",
+      "primary:092561",
+      "primary:182736",
+      "primary:158244",
+      "primary:152569",
+      "primary:011118"
+    ],
+    "threeway": null
+  },
+  "plots": {
+    "confusion_matrix": "/home/nathank/BITS/dev/Phishing Model/reports/confusion_matrix.png",
+    "roc_curve": "/home/nathank/BITS/dev/Phishing Model/reports/roc_curve.png",
+    "pr_curve": "/home/nathank/BITS/dev/Phishing Model/reports/pr_curve.png",
+    "eval_finetune_curves": "/home/nathank/BITS/dev/Phishing Model/reports/eval_finetune_curves.png",
+    "eval_pretrain_curves": "/home/nathank/BITS/dev/Phishing Model/reports/eval_pretrain_curves.png"
+  }
+}
diff --git a/presentation/reports/metrics_test3.json b/presentation/reports/metrics_test3.json
new file mode 100644
index 0000000000000000000000000000000000000000..ebe59db888c66bcb4a36d95e87c98e9d078d2a95
--- /dev/null
+++ b/presentation/reports/metrics_test3.json
@@ -0,0 +1,1669 @@
+{
+  "timestamp": "2026-10-09 10:46",
+  "run_name": "ft-focal",
+  "split": "test3",
+  "split_file": "/home/nathank/BITS/dev/Phishing Model/data/processed/test_holdout3.parquet",
+  "split_title": "three-way holdout",
+  "limit_rows": null,
+  "dev_run": false,
+  "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+  "splits_in_invocation": [
+    "test",
+    "test3"
+  ],
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 3
+  },
+  "num_labels": 3,
+  "params": 337444867,
+  "checkpoint_config": {
+    "tokenizer": "tokenizer.json",
+    "finetune": {
+      "run_name": "ft-focal",
+      "init": "pretrained:checkpoints/pretrain/large-24L/final",
+      "n_layers": 24,
+      "freeze_layers": 0,
+      "loss": "focal",
+      "focal_gamma": 2.0,
+      "lr": 3e-05,
+      "batch_size": 32,
+      "micro_batch": 16,
+      "epochs": 2,
+      "max_steps": null,
+      "eval_every": 1250,
+      "log_every": 20,
+      "eval_batch_size": 64,
+      "warmup_fraction": 0.06,
+      "weight_decay": 0.01,
+      "betas": [
+        0.9,
+        0.98
+      ],
+      "eps": 1e-06,
+      "grad_clip": 1.0,
+      "patience": 2,
+      "schedule": "linear_decay",
+      "selection_metric": "val_f1_unwanted",
+      "pooling": "cls",
+      "num_labels": 3,
+      "synthetic": "off",
+      "limit_rows": null,
+      "seed": 42,
+      "cpu": false
+    },
+    "best_epoch": 1,
+    "best_step": 8750,
+    "num_labels": 3,
+    "label_names": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "binary_rollup_name": "Unwanted (phishing or spam)",
+    "selection_metric": "val_f1_unwanted",
+    "selection_value": 0.9951084002657166,
+    "val_metrics": {
+      "accuracy": 0.9947003402250719,
+      "precision": 0.9954089646007007,
+      "recall": 0.9948080173871046,
+      "f1": 0.9951084002657166,
+      "roc_auc": 0.9998186946414248,
+      "pr_auc": 0.9998464119156345,
+      "tn": 6964,
+      "fp": 38,
+      "fn": 43,
+      "tp": 8239,
+      "threshold": 0.5,
+      "f1_optimal_threshold": 0.42074909806251526,
+      "f1_at_optimal_threshold": 0.9952995058454863,
+      "f1_unwanted": 0.9951084002657166,
+      "n_3way": 8105,
+      "accuracy_3way": 0.994694632942628,
+      "macro_f1_3way": 0.9868751212108838,
+      "precision_legitimate": 0.9988533753762362,
+      "recall_legitimate": 0.99528706083976,
+      "f1_legitimate": 0.9970670291151013,
+      "precision_phishing": 0.9853372434017595,
+      "recall_phishing": 0.9882352941176471,
+      "f1_phishing": 0.986784140969163,
+      "precision_spam": 0.9618805590851334,
+      "recall_spam": 0.9921363040629095,
+      "f1_spam": 0.9767741935483871,
+      "cm3_ll": 6969,
+      "cm3_lp": 4,
+      "cm3_ls": 29,
+      "cm3_pl": 3,
+      "cm3_pp": 336,
+      "cm3_ps": 1,
+      "cm3_sl": 5,
+      "cm3_sp": 1,
+      "cm3_ss": 757,
+      "n_unwanted_3way": 1103,
+      "pvs_phish_as_phish": 339,
+      "pvs_phish_as_spam": 1,
+      "pvs_spam_as_phish": 1,
+      "pvs_spam_as_spam": 762,
+      "val_loss": 0.030296484990695233,
+      "n": 15284,
+      "eval_seconds": 48.0
+    },
+    "val_metric_sets": {
+      "binary_unwanted": {
+        "accuracy": 0.9947003402250719,
+        "precision": 0.9954089646007007,
+        "recall": 0.9948080173871046,
+        "f1": 0.9951084002657166,
+        "roc_auc": 0.9998186946414248,
+        "pr_auc": 0.9998464119156345,
+        "tn": 6964,
+        "fp": 38,
+        "fn": 43,
+        "tp": 8239,
+        "threshold": 0.5,
+        "f1_optimal_threshold": 0.42074909806251526,
+        "f1_at_optimal_threshold": 0.9952995058454863,
+        "n": 15284,
+        "positive_class": "Unwanted (phishing or spam)"
+      },
+      "three_way": {
+        "n": 8105,
+        "accuracy": 0.994694632942628,
+        "macro_f1": 0.9868751212108838,
+        "per_class": {
+          "Legitimate": {
+            "precision": 0.9988533753762362,
+            "recall": 0.99528706083976,
+            "f1": 0.9970670291151013
+          },
+          "Phishing": {
+            "precision": 0.9853372434017595,
+            "recall": 0.9882352941176471,
+            "f1": 0.986784140969163
+          },
+          "Spam": {
+            "precision": 0.9618805590851334,
+            "recall": 0.9921363040629095,
+            "f1": 0.9767741935483871
+          }
+        },
+        "confusion": {
+          "order": [
+            "Legitimate",
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              6969,
+              4,
+              29
+            ],
+            [
+              3,
+              336,
+              1
+            ],
+            [
+              5,
+              1,
+              757
+            ]
+          ]
+        },
+        "phish_vs_spam_on_true_unwanted": {
+          "n": 1103,
+          "order": [
+            "Phishing",
+            "Spam"
+          ],
+          "rows_true_cols_pred": [
+            [
+              339,
+              1
+            ],
+            [
+              1,
+              762
+            ]
+          ]
+        }
+      }
+    },
+    "thresholds": {
+      "decision": 0.5,
+      "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+      "val_f1_optimal": 0.42074909806251526,
+      "val_f1_at_optimal": 0.9952995058454863
+    },
+    "init": "pretrained:/home/nathank/BITS/dev/Phishing Model/checkpoints/pretrain/large-24L/final",
+    "params": {
+      "total": 337444867,
+      "trainable": 337444867,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1052675
+    }
+  },
+  "tokenizer": "/home/nathank/BITS/dev/Phishing Model/data/processed/tokenizer.json",
+  "max_len": 512,
+  "label_names": {
+    "0": "Legitimate",
+    "1": "Phishing",
+    "2": "Spam"
+  },
+  "binary_rollup": {
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "classes": [
+      1,
+      2
+    ],
+    "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"
+  },
+  "split_sizes": {
+    "rows": 3220,
+    "positive": 1221,
+    "negative": 1999,
+    "positive_rate": 0.3792,
+    "label3": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "by_source": {
+      "Assassin": {
+        "rows": 236,
+        "positive": 72,
+        "label3": {
+          "legit": 164,
+          "phish": 0,
+          "spam": 72,
+          "unwanted": 0
+        }
+      },
+      "CEAS-08": {
+        "rows": 736,
+        "positive": 0,
+        "label3": {
+          "legit": 736,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Enron": {
+        "rows": 628,
+        "positive": 628,
+        "label3": {
+          "legit": 0,
+          "phish": 0,
+          "spam": 628,
+          "unwanted": 0
+        }
+      },
+      "Nazario": {
+        "rows": 222,
+        "positive": 222,
+        "label3": {
+          "legit": 0,
+          "phish": 222,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "Nigerian_Fraud": {
+        "rows": 299,
+        "positive": 299,
+        "label3": {
+          "legit": 0,
+          "phish": 299,
+          "spam": 0,
+          "unwanted": 0
+        }
+      },
+      "TREC-07": {
+        "rows": 1099,
+        "positive": 0,
+        "label3": {
+          "legit": 1099,
+          "phish": 0,
+          "spam": 0,
+          "unwanted": 0
+        }
+      }
+    },
+    "mean_tokens_full": 835.2,
+    "pct_truncated": 34.01
+  },
+  "threshold": 0.5,
+  "at_threshold": {
+    "threshold": 0.5,
+    "accuracy": 0.989751552795031,
+    "precision": 0.9892915980230642,
+    "recall": 0.9836199836199836,
+    "f1": 0.986447638603696,
+    "specificity": 0.9934967483741871,
+    "false_positive_rate": 0.0065032516258129065,
+    "false_negative_rate": 0.01638001638001638,
+    "confusion_matrix": {
+      "tn": 1986,
+      "fp": 13,
+      "fn": 20,
+      "tp": 1201
+    },
+    "n": 3220,
+    "n_positive": 1221,
+    "n_negative": 1999
+  },
+  "threshold_free": {
+    "roc_auc": 0.9993325901279878,
+    "pr_auc": 0.9988537821967886,
+    "recall_at_fpr_1pct": {
+      "fpr_budget": 0.01,
+      "recall": 0.9868959868959869,
+      "fpr": 0.009004502251125562,
+      "threshold": 0.43377745151519775,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "recall_at_fpr_0_1pct": {
+      "fpr_budget": 0.001,
+      "recall": 0.8787878787878788,
+      "fpr": 0.0005002501250625312,
+      "threshold": 0.9472352862358093,
+      "threshold_rule": "flag when p >= threshold"
+    },
+    "best_f1": {
+      "threshold": 0.47499534487724304,
+      "f1": 0.9876948318293682,
+      "precision": 0.9893179950698439,
+      "recall": 0.9860769860769861
+    }
+  },
+  "threeway": {
+    "num_labels": 3,
+    "label_names": {
+      "0": "Legitimate",
+      "1": "Phishing",
+      "2": "Spam"
+    },
+    "unwanted_name": "Unwanted (phishing or spam)",
+    "decision_rule": "rollup_then_argmax",
+    "decision_rule_doc": "unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.",
+    "threshold": 0.5,
+    "label3_counts": {
+      "legit": 1999,
+      "phish": 521,
+      "spam": 700,
+      "unwanted": 0
+    },
+    "predicted_kind_breakdown": {
+      "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+      "true_positives_flagged": {
+        "n": 1201,
+        "predicted_phishing": 515,
+        "predicted_spam": 686
+      },
+      "false_positives_flagged": {
+        "n": 13,
+        "predicted_phishing": 1,
+        "predicted_spam": 12
+      },
+      "missed_positives": {
+        "n": 20,
+        "predicted_phishing": 3,
+        "predicted_spam": 17
+      },
+      "all_flagged": {
+        "n": 1214,
+        "predicted_phishing": 516,
+        "predicted_spam": 698
+      },
+      "unsplit_unwanted_rows_flagged": {
+        "n": 0,
+        "predicted_phishing": 0,
+        "predicted_spam": 0
+      }
+    },
+    "rows_with_full_label": 3220,
+    "rows_unwanted_only": 0,
+    "evaluated": true,
+    "reason": null,
+    "confusion_matrix": {
+      "labels": [
+        "Legitimate",
+        "Phishing",
+        "Spam"
+      ],
+      "matrix": [
+        [
+          1986,
+          1,
+          12
+        ],
+        [
+          4,
+          514,
+          3
+        ],
+        [
+          16,
+          1,
+          683
+        ]
+      ],
+      "cells": {
+        "true_legitimate": {
+          "pred_legitimate": 1986,
+          "pred_phishing": 1,
+          "pred_spam": 12
+        },
+        "true_phishing": {
+          "pred_legitimate": 4,
+          "pred_phishing": 514,
+          "pred_spam": 3
+        },
+        "true_spam": {
+          "pred_legitimate": 16,
+          "pred_phishing": 1,
+          "pred_spam": 683
+        }
+      }
+    },
+    "accuracy": 0.9885093167701864,
+    "macro_f1": 0.9867305252015051,
+    "macro_f1_present_classes": 0.9867305252015051,
+    "classes_present": [
+      "Legitimate",
+      "Phishing",
+      "Spam"
+    ],
+    "per_class": {
+      "Legitimate": {
+        "precision": 0.9900299102691924,
+        "recall": 0.9934967483741871,
+        "f1": 0.9917602996254682,
+        "support": 1999
+      },
+      "Phishing": {
+        "precision": 0.9961240310077519,
+        "recall": 0.9865642994241842,
+        "f1": 0.991321118611379,
+        "support": 521
+      },
+      "Spam": {
+        "precision": 0.9785100286532952,
+        "recall": 0.9757142857142858,
+        "f1": 0.977110157367668,
+        "support": 700
+      }
+    },
+    "argmax_rule": {
+      "doc": "plain argmax over the three probabilities (secondary; can disagree with the binary verdict near the threshold)",
+      "accuracy": 0.987888198757764,
+      "macro_f1": 0.9858185605052249,
+      "per_class": {
+        "Legitimate": {
+          "precision": 0.9880715705765407,
+          "recall": 0.9944972486243121,
+          "f1": 0.9912739965095986,
+          "support": 1999
+        },
+        "Phishing": {
+          "precision": 0.9980430528375733,
+          "recall": 0.9788867562380038,
+          "f1": 0.9883720930232558,
+          "support": 521
+        },
+        "Spam": {
+          "precision": 0.9799139167862266,
+          "recall": 0.9757142857142858,
+          "f1": 0.9778095919828204,
+          "support": 700
+        }
+      },
+      "rows_where_rules_differ": 6
+    },
+    "phish_vs_spam_on_true_unwanted": {
+      "n_true_unwanted": 1221,
+      "kind_only": {
+        "doc": "given the row is unwanted, is argmax(P(phishing), P(spam)) right? (independent of the roll-up threshold)",
+        "labels": [
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            517,
+            4
+          ],
+          [
+            1,
+            699
+          ]
+        ],
+        "cells": {
+          "true_phishing": {
+            "pred_phishing": 517,
+            "pred_spam": 4
+          },
+          "true_spam": {
+            "pred_phishing": 1,
+            "pred_spam": 699
+          }
+        },
+        "accuracy": 0.9959049959049959,
+        "phishing_as_spam": 4,
+        "spam_as_phishing": 1
+      },
+      "rollup_rule": {
+        "doc": "rows = true Phishing / Spam; columns = predicted Legitimate (missed) / Phishing / Spam under the roll-up rule",
+        "rows": [
+          "Phishing",
+          "Spam"
+        ],
+        "columns": [
+          "Legitimate",
+          "Phishing",
+          "Spam"
+        ],
+        "matrix": [
+          [
+            4,
+            514,
+            3
+          ],
+          [
+            16,
+            1,
+            683
+          ]
+        ],
+        "missed_as_legitimate": {
+          "phishing": 4,
+          "spam": 16
+        }
+      }
+    },
+    "per_source": {
+      "Assassin": {
+        "n": 236,
+        "counts": {
+          "Legitimate": 164,
+          "Phishing": 0,
+          "Spam": 72
+        },
+        "accuracy": 0.9533898305084746,
+        "macro_f1": 0.6305609284332688,
+        "macro_f1_present_classes": 0.9458413926499033,
+        "per_class_recall": {
+          "Legitimate": 0.975609756097561,
+          "Spam": 0.9027777777777778
+        }
+      },
+      "CEAS-08": {
+        "n": 736,
+        "counts": {
+          "Legitimate": 736,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9904891304347826,
+        "macro_f1": 0.331740614334471,
+        "macro_f1_present_classes": 0.995221843003413,
+        "per_class_recall": {
+          "Legitimate": 0.9904891304347826
+        }
+      },
+      "Enron": {
+        "n": 628,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 0,
+          "Spam": 628
+        },
+        "accuracy": 0.9840764331210191,
+        "macro_f1": 0.3306581059390048,
+        "macro_f1_present_classes": 0.9919743178170144,
+        "per_class_recall": {
+          "Spam": 0.9840764331210191
+        }
+      },
+      "Nazario": {
+        "n": 222,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 222,
+          "Spam": 0
+        },
+        "accuracy": 0.9864864864864865,
+        "macro_f1": 0.3310657596371882,
+        "macro_f1_present_classes": 0.9931972789115646,
+        "per_class_recall": {
+          "Phishing": 0.9864864864864865
+        }
+      },
+      "Nigerian_Fraud": {
+        "n": 299,
+        "counts": {
+          "Legitimate": 0,
+          "Phishing": 299,
+          "Spam": 0
+        },
+        "accuracy": 0.9866220735785953,
+        "macro_f1": 0.33108866442199775,
+        "macro_f1_present_classes": 0.9932659932659933,
+        "per_class_recall": {
+          "Phishing": 0.9866220735785953
+        }
+      },
+      "TREC-07": {
+        "n": 1099,
+        "counts": {
+          "Legitimate": 1099,
+          "Phishing": 0,
+          "Spam": 0
+        },
+        "accuracy": 0.9981801637852593,
+        "macro_f1": 0.33302975106253796,
+        "macro_f1_present_classes": 0.9990892531876139,
+        "per_class_recall": {
+          "Legitimate": 0.9981801637852593
+        }
+      }
+    }
+  },
+  "credential_lure_slice": {
+    "n_phrases": 37,
+    "phrases": [
+      "verify your account",
+      "verify your identity",
+      "confirm your identity",
+      "confirm your account",
+      "enter your password",
+      "update your account",
+      "update your information",
+      "update your billing",
+      "update your payment",
+      "click here to verify",
+      "click the link below to verify",
+      "account has been suspended",
+      "account has been limited",
+      "account will be suspended",
+      "account has been locked",
+      "unusual sign-in",
+      "unusual activity",
+      "suspicious activity",
+      "paypal account",
+      "ebay account",
+      "password expire",
+      "password will expire",
+      "reset your password",
+      "invoice attached",
+      "wire transfer",
+      "gift card",
+      "urgent",
+      "urgently",
+      "security alert",
+      "security notice",
+      "login to your account",
+      "log in to your account",
+      "sign in to your account",
+      "bank account",
+      "failure to verify",
+      "within 24 hours",
+      "within 48 hours"
+    ],
+    "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+    "phrase_hits": {
+      "urgent": 139,
+      "bank account": 91,
+      "urgently": 25,
+      "verify your account": 15,
+      "confirm your account": 9,
+      "update your account": 9,
+      "within 24 hours": 8,
+      "security alert": 8,
+      "paypal account": 7,
+      "password will expire": 7,
+      "click here to verify": 6,
+      "unusual activity": 4,
+      "wire transfer": 3,
+      "update your information": 3,
+      "gift card": 3,
+      "verify your identity": 2,
+      "unusual sign-in": 2,
+      "account will be suspended": 2,
+      "update your payment": 2,
+      "failure to verify": 2,
+      "within 48 hours": 2,
+      "confirm your identity": 2,
+      "enter your password": 1,
+      "update your billing": 1,
+      "reset your password": 1,
+      "suspicious activity": 1,
+      "security notice": 1,
+      "log in to your account": 1
+    },
+    "phrase_hits_by_label": {
+      "verify your account": {
+        "positive": 15,
+        "negative": 0
+      },
+      "verify your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your identity": {
+        "positive": 2,
+        "negative": 0
+      },
+      "confirm your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "enter your password": {
+        "positive": 0,
+        "negative": 1
+      },
+      "update your account": {
+        "positive": 9,
+        "negative": 0
+      },
+      "update your information": {
+        "positive": 2,
+        "negative": 1
+      },
+      "update your billing": {
+        "positive": 1,
+        "negative": 0
+      },
+      "update your payment": {
+        "positive": 2,
+        "negative": 0
+      },
+      "click here to verify": {
+        "positive": 6,
+        "negative": 0
+      },
+      "account will be suspended": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual sign-in": {
+        "positive": 2,
+        "negative": 0
+      },
+      "unusual activity": {
+        "positive": 4,
+        "negative": 0
+      },
+      "suspicious activity": {
+        "positive": 1,
+        "negative": 0
+      },
+      "paypal account": {
+        "positive": 7,
+        "negative": 0
+      },
+      "password will expire": {
+        "positive": 7,
+        "negative": 0
+      },
+      "reset your password": {
+        "positive": 1,
+        "negative": 0
+      },
+      "wire transfer": {
+        "positive": 2,
+        "negative": 1
+      },
+      "gift card": {
+        "positive": 3,
+        "negative": 0
+      },
+      "urgent": {
+        "positive": 133,
+        "negative": 6
+      },
+      "urgently": {
+        "positive": 25,
+        "negative": 0
+      },
+      "security alert": {
+        "positive": 8,
+        "negative": 0
+      },
+      "security notice": {
+        "positive": 1,
+        "negative": 0
+      },
+      "log in to your account": {
+        "positive": 1,
+        "negative": 0
+      },
+      "bank account": {
+        "positive": 91,
+        "negative": 0
+      },
+      "failure to verify": {
+        "positive": 2,
+        "negative": 0
+      },
+      "within 24 hours": {
+        "positive": 8,
+        "negative": 0
+      },
+      "within 48 hours": {
+        "positive": 2,
+        "negative": 0
+      }
+    },
+    "label_purity": 0.9745,
+    "min_positives_for_comparison": 20,
+    "label3_counts": {
+      "legit": 7,
+      "phish": 236,
+      "spam": 32,
+      "unwanted": 0
+    },
+    "threeway_accuracy": 0.9927272727272727,
+    "metrics": {
+      "threshold": 0.5,
+      "accuracy": 1.0,
+      "precision": 1.0,
+      "recall": 1.0,
+      "f1": 1.0,
+      "specificity": 1.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 7,
+        "fp": 0,
+        "fn": 0,
+        "tp": 268
+      },
+      "n": 275,
+      "n_positive": 268,
+      "n_negative": 7,
+      "roc_auc": 1.0
+    }
+  },
+  "per_source": {
+    "Assassin": {
+      "threshold": 0.5,
+      "accuracy": 0.9576271186440678,
+      "precision": 0.9428571428571428,
+      "recall": 0.9166666666666666,
+      "f1": 0.9295774647887323,
+      "specificity": 0.975609756097561,
+      "false_positive_rate": 0.024390243902439025,
+      "false_negative_rate": 0.08333333333333333,
+      "confusion_matrix": {
+        "tn": 160,
+        "fp": 4,
+        "fn": 6,
+        "tp": 66
+      },
+      "n": 236,
+      "n_positive": 72,
+      "n_negative": 164,
+      "roc_auc": 0.994579945799458,
+      "label3_counts": {
+        "legit": 164,
+        "phish": 0,
+        "spam": 72,
+        "unwanted": 0
+      }
+    },
+    "CEAS-08": {
+      "threshold": 0.5,
+      "accuracy": 0.9904891304347826,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9904891304347826,
+      "false_positive_rate": 0.009510869565217392,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 729,
+        "fp": 7,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 736,
+      "n_positive": 0,
+      "n_negative": 736,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 736,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Enron": {
+      "threshold": 0.5,
+      "accuracy": 0.9840764331210191,
+      "precision": 1.0,
+      "recall": 0.9840764331210191,
+      "f1": 0.9919743178170145,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.01592356687898089,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 10,
+        "tp": 618
+      },
+      "n": 628,
+      "n_positive": 628,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 0,
+        "spam": 628,
+        "unwanted": 0
+      }
+    },
+    "Nazario": {
+      "threshold": 0.5,
+      "accuracy": 0.9864864864864865,
+      "precision": 1.0,
+      "recall": 0.9864864864864865,
+      "f1": 0.9931972789115647,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.013513513513513514,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 3,
+        "tp": 219
+      },
+      "n": 222,
+      "n_positive": 222,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 222,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "Nigerian_Fraud": {
+      "threshold": 0.5,
+      "accuracy": 0.9966555183946488,
+      "precision": 1.0,
+      "recall": 0.9966555183946488,
+      "f1": 0.998324958123953,
+      "specificity": 0.0,
+      "false_positive_rate": 0.0,
+      "false_negative_rate": 0.0033444816053511705,
+      "confusion_matrix": {
+        "tn": 0,
+        "fp": 0,
+        "fn": 1,
+        "tp": 298
+      },
+      "n": 299,
+      "n_positive": 299,
+      "n_negative": 0,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 0,
+        "phish": 299,
+        "spam": 0,
+        "unwanted": 0
+      }
+    },
+    "TREC-07": {
+      "threshold": 0.5,
+      "accuracy": 0.9981801637852593,
+      "precision": 0.0,
+      "recall": 0.0,
+      "f1": 0.0,
+      "specificity": 0.9981801637852593,
+      "false_positive_rate": 0.0018198362147406734,
+      "false_negative_rate": 0.0,
+      "confusion_matrix": {
+        "tn": 1097,
+        "fp": 2,
+        "fn": 0,
+        "tp": 0
+      },
+      "n": 1099,
+      "n_positive": 0,
+      "n_negative": 1099,
+      "roc_auc": null,
+      "label3_counts": {
+        "legit": 1099,
+        "phish": 0,
+        "spam": 0,
+        "unwanted": 0
+      }
+    }
+  },
+  "length_confound": {
+    "by_outcome": {
+      "tp": {
+        "n": 1201,
+        "mean_tokens_full": 978.0,
+        "mean_tokens_fed": 279.8,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.6
+      },
+      "tn": {
+        "n": 1986,
+        "mean_tokens_full": 732.6,
+        "mean_tokens_fed": 358.4,
+        "median_tokens_full": 392.5,
+        "pct_truncated": 38.3
+      },
+      "fp": {
+        "n": 13,
+        "mean_tokens_full": 1814.3,
+        "mean_tokens_fed": 377.2,
+        "median_tokens_full": 875.0,
+        "pct_truncated": 61.5
+      },
+      "fn": {
+        "n": 20,
+        "mean_tokens_full": 1807.8,
+        "mean_tokens_fed": 284.1,
+        "median_tokens_full": 291.0,
+        "pct_truncated": 35.0
+      },
+      "correct": {
+        "n": 3187,
+        "mean_tokens_full": 825.1,
+        "mean_tokens_fed": 328.8,
+        "median_tokens_full": 339.0,
+        "pct_truncated": 33.9
+      },
+      "wrong": {
+        "n": 33,
+        "mean_tokens_full": 1810.4,
+        "mean_tokens_fed": 320.8,
+        "median_tokens_full": 368.0,
+        "pct_truncated": 45.5
+      }
+    },
+    "by_label": {
+      "legitimate": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "unwanted": {
+        "n": 1221,
+        "mean_tokens_full": 991.6,
+        "mean_tokens_fed": 279.9,
+        "median_tokens_full": 243.0,
+        "pct_truncated": 26.7
+      }
+    },
+    "by_class3": {
+      "legit": {
+        "n": 1999,
+        "mean_tokens_full": 739.7,
+        "mean_tokens_fed": 358.6,
+        "median_tokens_full": 393.0,
+        "pct_truncated": 38.5
+      },
+      "phish": {
+        "n": 521,
+        "mean_tokens_full": 1349.4,
+        "mean_tokens_fed": 355.1,
+        "median_tokens_full": 412.0,
+        "pct_truncated": 39.9
+      },
+      "spam": {
+        "n": 700,
+        "mean_tokens_full": 725.3,
+        "mean_tokens_fed": 223.9,
+        "median_tokens_full": 159.0,
+        "pct_truncated": 16.9
+      }
+    },
+    "by_length_bucket": [
+      {
+        "bucket": "<=64",
+        "n": 129,
+        "n_positive": 99,
+        "positive_rate": 0.7674418604651163,
+        "recall": 0.9494949494949495,
+        "fpr": 0.03333333333333333,
+        "accuracy": 0.9534883720930233
+      },
+      {
+        "bucket": "65-128",
+        "n": 427,
+        "n_positive": 252,
+        "positive_rate": 0.5901639344262295,
+        "recall": 0.9920634920634921,
+        "fpr": 0.011428571428571429,
+        "accuracy": 0.990632318501171
+      },
+      {
+        "bucket": "129-256",
+        "n": 699,
+        "n_positive": 276,
+        "positive_rate": 0.3948497854077253,
+        "recall": 0.9927536231884058,
+        "fpr": 0.0,
+        "accuracy": 0.9971387696709585
+      },
+      {
+        "bucket": "257-512",
+        "n": 870,
+        "n_positive": 268,
+        "positive_rate": 0.3080459770114943,
+        "recall": 0.9850746268656716,
+        "fpr": 0.0033222591362126247,
+        "accuracy": 0.993103448275862
+      },
+      {
+        "bucket": ">512",
+        "n": 1095,
+        "n_positive": 326,
+        "positive_rate": 0.29771689497716897,
+        "recall": 0.9785276073619632,
+        "fpr": 0.010403120936280884,
+        "accuracy": 0.9863013698630136
+      }
+    ]
+  },
+  "inference": {
+    "device": "cuda",
+    "autocast_bf16": true,
+    "batch_size": 64,
+    "pad_to_multiple_of": 8,
+    "num_labels": 3,
+    "rows": 3220,
+    "seconds": 13.55,
+    "rows_per_s": 237.6,
+    "padded_tokens": 1079936,
+    "real_tokens": 1058490,
+    "padded_tokens_per_s": 79672,
+    "peak_vram_gb": 2.27
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "samples_requested": 100,
+    "sample_split": "test",
+    "max_len": 512,
+    "gpu_bf16": {
+      "n": 100,
+      "avg_ms": 10.08,
+      "p50_ms": 8.9,
+      "p95_ms": 13.72,
+      "max_ms": 18.58,
+      "avg_s": 0.0101,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 0.136,
+      "device": "NVIDIA GeForce RTX 3090 Ti"
+    },
+    "cpu_fp32": {
+      "n": 100,
+      "avg_ms": 458.57,
+      "p50_ms": 445.01,
+      "p95_ms": 701.07,
+      "max_ms": 828.65,
+      "avg_s": 0.4586,
+      "mean_tokens": 280.0,
+      "ratio_to_benchmark": 6.197,
+      "threads": 16,
+      "estimated_full_run_s": 41.8
+    },
+    "checkpoint": "/home/nathank/BITS/dev/Phishing Model/checkpoints/finetune/ft-focal/best",
+    "run_name": "ft-focal",
+    "params": 337444867,
+    "timestamp": "2026-10-09 10:46",
+    "num_labels": 3,
+    "measured_once_for_splits": [
+      "test",
+      "test3"
+    ]
+  },
+  "error_analysis": {
+    "path": "/home/nathank/BITS/dev/Phishing Model/reports/error_analysis_test3.md",
+    "n_fn": 20,
+    "n_fp": 13,
+    "composition": {
+      "fp": {
+        "n": 13,
+        "mailing_list_or_digest": 0.462,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.615,
+        "lure_phrase": 0.0
+      },
+      "tn": {
+        "n": 1986,
+        "mailing_list_or_digest": 0.563,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.0,
+        "truncated_gt512": 0.383,
+        "lure_phrase": 0.004
+      },
+      "fn": {
+        "n": 20,
+        "mailing_list_or_digest": 0.3,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.1,
+        "truncated_gt512": 0.35,
+        "lure_phrase": 0.0
+      },
+      "tp": {
+        "n": 1201,
+        "mailing_list_or_digest": 0.083,
+        "html_heavy": 0.0,
+        "very_short_lt20_tokens": 0.006,
+        "truncated_gt512": 0.266,
+        "lure_phrase": 0.223
+      }
+    },
+    "fn_vs_tp_tokens": [
+      {
+        "token": "invite",
+        "share_a": 0.25,
+        "share_b": 0.002,
+        "ratio": 128.94
+      },
+      {
+        "token": "cut",
+        "share_a": 0.15,
+        "share_b": 0.002,
+        "ratio": 82.05
+      },
+      {
+        "token": "practical",
+        "share_a": 0.15,
+        "share_b": 0.002,
+        "ratio": 82.05
+      },
+      {
+        "token": "networking",
+        "share_a": 0.2,
+        "share_b": 0.002,
+        "ratio": 75.36
+      },
+      {
+        "token": "audience",
+        "share_a": 0.15,
+        "share_b": 0.003,
+        "ratio": 45.59
+      },
+      {
+        "token": "oct",
+        "share_a": 0.15,
+        "share_b": 0.003,
+        "ratio": 45.59
+      },
+      {
+        "token": "innovative",
+        "share_a": 0.15,
+        "share_b": 0.003,
+        "ratio": 45.59
+      },
+      {
+        "token": "event",
+        "share_a": 0.2,
+        "share_b": 0.005,
+        "ratio": 40.58
+      },
+      {
+        "token": "angeles",
+        "share_a": 0.15,
+        "share_b": 0.004,
+        "ratio": 37.3
+      },
+      {
+        "token": "anyway",
+        "share_a": 0.15,
+        "share_b": 0.004,
+        "ratio": 37.3
+      },
+      {
+        "token": "feature",
+        "share_a": 0.15,
+        "share_b": 0.005,
+        "ratio": 31.56
+      },
+      {
+        "token": "york",
+        "share_a": 0.15,
+        "share_b": 0.005,
+        "ratio": 31.56
+      },
+      {
+        "token": "subscription",
+        "share_a": 0.2,
+        "share_b": 0.007,
+        "ratio": 27.76
+      },
+      {
+        "token": "technologies",
+        "share_a": 0.15,
+        "share_b": 0.006,
+        "ratio": 27.35
+      },
+      {
+        "token": "entertainment",
+        "share_a": 0.15,
+        "share_b": 0.007,
+        "ratio": 24.13
+      },
+      {
+        "token": "staff",
+        "share_a": 0.2,
+        "share_b": 0.011,
+        "ratio": 19.54
+      },
+      {
+        "token": "association",
+        "share_a": 0.15,
+        "share_b": 0.008,
+        "ratio": 19.54
+      },
+      {
+        "token": "markets",
+        "share_a": 0.15,
+        "share_b": 0.009,
+        "ratio": 17.84
+      },
+      {
+        "token": "hosting",
+        "share_a": 0.15,
+        "share_b": 0.009,
+        "ratio": 17.84
+      },
+      {
+        "token": "mailman",
+        "share_a": 0.15,
+        "share_b": 0.009,
+        "ratio": 17.84
+      }
+    ],
+    "fp_vs_tn_tokens": [
+      {
+        "token": "outlets",
+        "share_a": 0.231,
+        "share_b": 0.001,
+        "ratio": 206.01
+      },
+      {
+        "token": "zzz-unspun",
+        "share_a": 0.231,
+        "share_b": 0.001,
+        "ratio": 206.01
+      },
+      {
+        "token": "czzz-unspun",
+        "share_a": 0.231,
+        "share_b": 0.001,
+        "ratio": 206.01
+      },
+      {
+        "token": "cook",
+        "share_a": 0.231,
+        "share_b": 0.001,
+        "ratio": 206.01
+      },
+      {
+        "token": "full-color",
+        "share_a": 0.385,
+        "share_b": 0.002,
+        "ratio": 179.85
+      },
+      {
+        "token": "muehlbauer",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "vanderkolk",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "eletra",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "loripatel",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "copyedited",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "jimduffy86",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "asbrand",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "patel",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "authoritatively",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "four-week",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "duffy",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "mediaunspun",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "savviest",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "dasbrand",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      },
+      {
+        "token": "sendashtml",
+        "share_a": 0.231,
+        "share_b": 0.002,
+        "ratio": 147.15
+      }
+    ],
+    "top_fn_doc_ids": [
+      "primary:191966",
+      "primary:148887",
+      "aura:Nazario:000097",
+      "primary:010013",
+      "primary:007176",
+      "aura:Nazario:000643",
+      "primary:181246",
+      "primary:095405",
+      "primary:014300",
+      "primary:001939",
+      "primary:033220",
+      "primary:025244",
+      "primary:133074",
+      "aura:Nigerian_Fraud:003049",
+      "aura:Nazario:000299",
+      "primary:046569",
+      "primary:135631",
+      "primary:148857",
+      "primary:129677",
+      "primary:075760"
+    ],
+    "top_fp_doc_ids": [
+      "primary:144048",
+      "primary:199386",
+      "primary:091801",
+      "primary:152763",
+      "primary:165228",
+      "primary:041713",
+      "primary:103927",
+      "primary:055965",
+      "primary:170237",
+      "primary:117398",
+      "primary:168120",
+      "primary:057514",
+      "primary:021911"
+    ],
+    "threeway": {
+      "n_phish_as_spam": 3,
+      "n_spam_as_phish": 1,
+      "top_phish_as_spam_doc_ids": [
+        "aura:Nigerian_Fraud:001698",
+        "aura:Nigerian_Fraud:000337",
+        "aura:Nigerian_Fraud:000824"
+      ],
+      "top_spam_as_phish_doc_ids": [
+        "primary:077835"
+      ],
+      "phish_as_spam_tokens": [
+        {
+          "token": "join",
+          "share_a": 1.0,
+          "share_b": 0.019,
+          "ratio": 49.0
+        },
+        {
+          "token": "could",
+          "share_a": 1.0,
+          "share_b": 0.076,
+          "ratio": 13.03
+        },
+        {
+          "token": "www",
+          "share_a": 1.0,
+          "share_b": 0.15,
+          "ratio": 6.64
+        },
+        {
+          "token": "new",
+          "share_a": 1.0,
+          "share_b": 0.16,
+          "ratio": 6.24
+        },
+        {
+          "token": "http",
+          "share_a": 1.0,
+          "share_b": 0.193,
+          "ratio": 5.17
+        },
+        {
+          "token": "years",
+          "share_a": 1.0,
+          "share_b": 0.196,
+          "ratio": 5.07
+        },
+        {
+          "token": "get",
+          "share_a": 1.0,
+          "share_b": 0.202,
+          "ratio": 4.92
+        },
+        {
+          "token": "about",
+          "share_a": 1.0,
+          "share_b": 0.239,
+          "ratio": 4.17
+        },
+        {
+          "token": "after",
+          "share_a": 1.0,
+          "share_b": 0.278,
+          "ratio": 3.59
+        },
+        {
+          "token": "who",
+          "share_a": 1.0,
+          "share_b": 0.317,
+          "ratio": 3.15
+        },
+        {
+          "token": "into",
+          "share_a": 1.0,
+          "share_b": 0.323,
+          "ratio": 3.09
+        },
+        {
+          "token": "com",
+          "share_a": 1.0,
+          "share_b": 0.339,
+          "ratio": 2.95
+        },
+        {
+          "token": "which",
+          "share_a": 1.0,
+          "share_b": 0.362,
+          "ratio": 2.76
+        },
+        {
+          "token": "now",
+          "share_a": 1.0,
+          "share_b": 0.379,
+          "ratio": 2.63
+        },
+        {
+          "token": "can",
+          "share_a": 1.0,
+          "share_b": 0.467,
+          "ratio": 2.14
+        },
+        {
+          "token": "are",
+          "share_a": 1.0,
+          "share_b": 0.562,
+          "ratio": 1.78
+        },
+        {
+          "token": "from",
+          "share_a": 1.0,
+          "share_b": 0.644,
+          "ratio": 1.55
+        },
+        {
+          "token": "will",
+          "share_a": 1.0,
+          "share_b": 0.714,
+          "ratio": 1.4
+        },
+        {
+          "token": "for",
+          "share_a": 1.0,
+          "share_b": 0.8,
+          "ratio": 1.25
+        },
+        {
+          "token": "this",
+          "share_a": 1.0,
+          "share_b": 0.815,
+          "ratio": 1.23
+        }
+      ],
+      "spam_as_phish_tokens": []
+    }
+  },
+  "plots": {
+    "confusion_matrix": "/home/nathank/BITS/dev/Phishing Model/reports/confusion_matrix_test3.png",
+    "confusion_matrix3": "/home/nathank/BITS/dev/Phishing Model/reports/confusion_matrix3_test3.png",
+    "roc_curve": "/home/nathank/BITS/dev/Phishing Model/reports/roc_curve_test3.png",
+    "pr_curve": "/home/nathank/BITS/dev/Phishing Model/reports/pr_curve_test3.png",
+    "eval_finetune_curves": "/home/nathank/BITS/dev/Phishing Model/reports/eval_finetune_curves_test3.png",
+    "eval_pretrain_curves": "/home/nathank/BITS/dev/Phishing Model/reports/eval_pretrain_curves_test3.png"
+  }
+}
diff --git a/presentation/reports/pr_curve.png b/presentation/reports/pr_curve.png
new file mode 100644
index 0000000000000000000000000000000000000000..f696f315ee16e045ce9a3bec7ef96194162c359e
Binary files /dev/null and b/presentation/reports/pr_curve.png differ
diff --git a/presentation/reports/pr_curve_test3.png b/presentation/reports/pr_curve_test3.png
new file mode 100644
index 0000000000000000000000000000000000000000..9c6ba8d85ea637ff7fab73306ba41fd40c389b89
Binary files /dev/null and b/presentation/reports/pr_curve_test3.png differ
diff --git a/presentation/reports/pretrain_curves.png b/presentation/reports/pretrain_curves.png
new file mode 100644
index 0000000000000000000000000000000000000000..ed60ba340c513ca86bacdbe3ddf22caf5d4790e1
--- /dev/null
+++ b/presentation/reports/pretrain_curves.png
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:362ffb969cc014cbbf339e94d61af8a903c8429080b70b6229de77cebc3fb65d
+size 117837
diff --git a/presentation/reports/roc_curve.png b/presentation/reports/roc_curve.png
new file mode 100644
index 0000000000000000000000000000000000000000..d24de7dc3d0c77f4152ee89f0b1ef9867ce4d3b1
Binary files /dev/null and b/presentation/reports/roc_curve.png differ
diff --git a/presentation/reports/roc_curve_test3.png b/presentation/reports/roc_curve_test3.png
new file mode 100644
index 0000000000000000000000000000000000000000..d29da937427d1ab450f6b12bce17470397bf385f
Binary files /dev/null and b/presentation/reports/roc_curve_test3.png differ
diff --git a/presentation/reports/smoke_test.json b/presentation/reports/smoke_test.json
new file mode 100644
index 0000000000000000000000000000000000000000..bbe4689570373567ce27a6e14d6cf97179c42fdb
--- /dev/null
+++ b/presentation/reports/smoke_test.json
@@ -0,0 +1,316 @@
+{
+  "timestamp": "2026-10-07 23:46:47",
+  "device": {
+    "python": "3.14.7",
+    "torch": "2.14.1+cu130",
+    "cuda_build": "13.0",
+    "cudnn": 92400,
+    "device": "NVIDIA GeForce RTX 3090 Ti",
+    "sm": "8.6",
+    "vram_gib": 23.6,
+    "bf16": true,
+    "free_gib": 21.52
+  },
+  "alloc_conf": "expandable_segments:True",
+  "baseline_device_used_gib": 2.08,
+  "model_config": {
+    "vocab_size": 32768,
+    "max_seq_len": 512,
+    "d_model": 1024,
+    "n_layers": 24,
+    "n_heads": 16,
+    "d_ff": 4096,
+    "dropout": 0.1,
+    "attention_dropout": 0.1,
+    "layer_norm_eps": 1e-05,
+    "pad_token_id": 0,
+    "tie_mlm_head": true,
+    "grad_checkpointing": false,
+    "num_labels": 2
+  },
+  "param_counts": {
+    "mlm": {
+      "total": 337476608,
+      "trainable": 337476608,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1084416
+    },
+    "classifier": {
+      "total": 337443842,
+      "trainable": 337443842,
+      "embeddings": 34080768,
+      "encoder_layers": 302309376,
+      "final_norm": 2048,
+      "head": 1051650
+    }
+  },
+  "data": {
+    "source": "real",
+    "detail": "3000 docs (6,139,250 chars) -> 1,592,926 tokens -> 3111 packed x 512",
+    "n_sequences": 3111,
+    "seq_len": 512,
+    "mlm_probability": 0.15,
+    "mask_replace_probs": [
+      0.8,
+      0.1,
+      0.1
+    ]
+  },
+  "vram_policy": {
+    "target_gb_low": 14,
+    "target_gb_high": 18,
+    "hard_cap_gb": 20,
+    "alloc_conf": "expandable_segments:True",
+    "log_peak_every_steps": 20
+  },
+  "vram_policy_unit": "config vram.*_gb values are read as GiB and compared with torch.cuda.max_memory_reserved() / 2**30",
+  "finder": {
+    "candidates": [
+      16,
+      24,
+      12,
+      8
+    ],
+    "ckpt_candidates": [
+      16,
+      24,
+      12,
+      8
+    ],
+    "probe_steps": 3,
+    "micro_steps_per_probe_step": 2,
+    "no_ckpt": [
+      {
+        "micro_batch": 16,
+        "grad_checkpointing": false,
+        "ok": true,
+        "steps": 3,
+        "micro_steps": 2,
+        "max_allocated_gib": 13.524,
+        "max_reserved_gib": 13.654,
+        "device_used_after_gib": 15.802,
+        "ms_per_micro_step": 346.5,
+        "tokens_per_s": 23642,
+        "error": null
+      },
+      {
+        "micro_batch": 24,
+        "grad_checkpointing": false,
+        "ok": true,
+        "steps": 3,
+        "micro_steps": 2,
+        "max_allocated_gib": 17.44,
+        "max_reserved_gib": 17.65,
+        "device_used_after_gib": 19.796,
+        "ms_per_micro_step": 487.1,
+        "tokens_per_s": 25227,
+        "error": null
+      },
+      {
+        "micro_batch": 12,
+        "grad_checkpointing": false,
+        "ok": true,
+        "steps": 3,
+        "micro_steps": 2,
+        "max_allocated_gib": 11.571,
+        "max_reserved_gib": 11.775,
+        "device_used_after_gib": 13.921,
+        "ms_per_micro_step": 261.6,
+        "tokens_per_s": 23487,
+        "error": null
+      },
+      {
+        "micro_batch": 8,
+        "grad_checkpointing": false,
+        "ok": true,
+        "steps": 3,
+        "micro_steps": 2,
+        "max_allocated_gib": 9.593,
+        "max_reserved_gib": 9.705,
+        "device_used_after_gib": 11.851,
+        "ms_per_micro_step": 190.6,
+        "tokens_per_s": 21493,
+        "error": null
+      }
+    ],
+    "ckpt": [
+      {
+        "micro_batch": 16,
+        "grad_checkpointing": true,
+        "ok": true,
+        "steps": 3,
+        "micro_steps": 2,
+        "max_allocated_gib": 6.82,
+        "max_reserved_gib": 7.381,
+        "device_used_after_gib": 9.526,
+        "ms_per_micro_step": 457.6,
+        "tokens_per_s": 17901,
+        "error": null
+      },
+      {
+        "micro_batch": 24,
+        "grad_checkpointing": true,
+        "ok": true,
+        "steps": 3,
+        "micro_steps": 2,
+        "max_allocated_gib": 7.396,
+        "max_reserved_gib": 8.064,
+        "device_used_after_gib": 10.21,
+        "ms_per_micro_step": 641.5,
+        "tokens_per_s": 19156,
+        "error": null
+      },
+      {
+        "micro_batch": 12,
+        "grad_checkpointing": true,
+        "ok": true,
+        "steps": 3,
+        "micro_steps": 2,
+        "max_allocated_gib": 6.538,
+        "max_reserved_gib": 6.932,
+        "device_used_after_gib": 9.077,
+        "ms_per_micro_step": 342.9,
+        "tokens_per_s": 17916,
+        "error": null
+      },
+      {
+        "micro_batch": 8,
+        "grad_checkpointing": true,
+        "ok": true,
+        "steps": 3,
+        "micro_steps": 2,
+        "max_allocated_gib": 6.256,
+        "max_reserved_gib": 6.58,
+        "device_used_after_gib": 8.728,
+        "ms_per_micro_step": 247.2,
+        "tokens_per_s": 16573,
+        "error": null
+      }
+    ],
+    "chosen_micro_batch": 24,
+    "reason": "largest candidate with reserved peak in [14, 18] GiB (17.65 GiB)",
+    "chosen_micro_batch_ckpt": 24,
+    "reason_ckpt": "no candidate reached 14 GiB; largest under 18 GiB (8.064 GiB)",
+    "throughput_peak_reserved_gib": 17.65,
+    "throughput_within_hard_cap": true,
+    "throughput_in_target_band": true
+  },
+  "throughput": {
+    "micro_batch": 24,
+    "grad_accumulation": 11,
+    "effective_batch": 264,
+    "effective_batch_target": 256,
+    "effective_batch_matches_target": false,
+    "peak_reserved_gib": 17.65,
+    "peak_allocated_gib": 17.448,
+    "timed_micro_steps": 20,
+    "warmup_micro_steps": 3,
+    "ms_per_micro_step_raw": 487.15,
+    "ms_fwd_bwd": 486.26,
+    "ms_optimizer_step": 17.78,
+    "ms_per_optimizer_step_total": 5366.7,
+    "tokens_per_s_raw": 25224,
+    "tokens_per_s_effective": 25187,
+    "epoch_tokens": 105836752.22169875,
+    "optimizer_steps_per_epoch": 783,
+    "hours_per_epoch": 1.17,
+    "hours_3_epochs": 3.5,
+    "vram": {
+      "allocated_gb": 3.799478054046631,
+      "reserved_gb": 4.177734375,
+      "max_allocated_gb": 17.447842597961426,
+      "max_reserved_gb": 17.650390625
+    },
+    "epoch_tokens_source": "splits_manifest.json 407,902,364 chars x 0.2595 tok/char"
+  },
+  "sdpa_backends": {
+    "flash_no_mask": "ran",
+    "efficient_bool_mask": "ran",
+    "flash_bool_mask_expected_fail": "unavailable: No available kernel. Aborting execution.",
+    "flags": {
+      "flash_sdp_enabled": true,
+      "mem_efficient_sdp_enabled": true,
+      "math_sdp_enabled": true
+    }
+  },
+  "convergence": {
+    "micro_batch": 24,
+    "steps": 30,
+    "lr": 0.0001,
+    "warmup_steps": 10,
+    "ln_vocab": 10.3972,
+    "first_loss": 10.5846,
+    "last_loss": 7.6686,
+    "drop": 2.916,
+    "losses": [
+      [
+        1,
+        10.5846
+      ],
+      [
+        5,
+        9.6535
+      ],
+      [
+        10,
+        8.9505
+      ],
+      [
+        15,
+        8.6266
+      ],
+      [
+        20,
+        8.1493
+      ],
+      [
+        25,
+        7.9209
+      ],
+      [
+        30,
+        7.6686
+      ]
+    ],
+    "data": "real",
+    "vram": {
+      "allocated_gb": 3.799478054046631,
+      "reserved_gb": 4.19921875,
+      "max_allocated_gb": 17.447842597961426,
+      "max_reserved_gb": 17.650390625
+    }
+  },
+  "latency": {
+    "instructor_benchmark_ms": 74.0,
+    "params": 337443842,
+    "gpu_bf16_512_no_mask": {
+      "n": 100,
+      "avg_ms": 10.7,
+      "p50_ms": 10.71,
+      "max_ms": 11.27
+    },
+    "gpu_bf16_512_bool_mask": {
+      "n": 100,
+      "avg_ms": 11.53,
+      "p50_ms": 11.51,
+      "max_ms": 12.14
+    },
+    "cpu_threads": 16,
+    "cpu_fp32_512": {
+      "n": 10,
+      "avg_ms": 698.8,
+      "p50_ms": 695.6,
+      "max_ms": 753.9
+    },
+    "cpu_fp32_128": {
+      "n": 10,
+      "avg_ms": 319.0,
+      "p50_ms": 320.6,
+      "max_ms": 345.1
+    }
+  },
+  "runtime_s": 76.1,
+  "report_path": "/home/nathank/BITS/dev/Phishing Model/reports/smoke_test.json"
+}
diff --git a/presentation/requirements-train.txt b/presentation/requirements-train.txt
new file mode 100644
index 0000000000000000000000000000000000000000..2d2a82bc2a64851262864b1044b285c2f8509b5b
--- /dev/null
+++ b/presentation/requirements-train.txt
@@ -0,0 +1,25 @@
+# Local training venv — Python 3.14.7 (system), RTX 3090 Ti, driver 615.71.09 (CUDA 13.4)
+# Verified live on 2026-10-07: every package has a cp314/abi3/pure Linux x86_64 wheel.
+# Install: uv pip install --python .venv/bin/python --torch-backend=cu130 -r requirements-train.txt
+# (never --torch-backend=auto: uv 0.12.5 would pick cu132 on this driver)
+torch==2.14.1
+tokenizers==0.23.2
+safetensors==0.8.0
+numpy==2.5.3
+huggingface_hub==1.33.0
+datasets==5.1.0
+fsspec==2026.7.0
+pyarrow==25.0.1
+pandas==3.0.6
+scipy==1.18.1
+scikit-learn==1.9.1
+matplotlib==3.11.2
+pyyaml==6.0.3
+tqdm==4.70.1
+jupyterlab==4.6.4
+ipykernel==7.4.0
+nbformat==5.11.1
+nbconvert==7.17.1
+ipywidgets==8.1.9
+gradio==6.29.1
+spaces==0.51.3
diff --git a/presentation/scripts/build_notebook.py b/presentation/scripts/build_notebook.py
new file mode 100644
index 0000000000000000000000000000000000000000..58b592d497a31b4503fea37b94bdd11d80860156
--- /dev/null
+++ b/presentation/scripts/build_notebook.py
@@ -0,0 +1,959 @@
+#!/usr/bin/env python
+"""Build FinalProject_PhishingDetector.ipynb with nbformat (idempotent: same cells every run).
+
+The notebook mirrors the instructor's reference (reference/FashionMNIST_Gradio_Full.ipynb): a
+title cell, then EXACTLY his nine bold "**Step N: ...**" markdown headers in his order, each
+followed by a short explanation and code cells that load OUR artifacts, then "Results" and
+"Challenges and what we learned". Every code cell runs on the CPU today: the first cell sets
+``USE_GPU = False`` (hides the GPU before torch is imported, so opening the notebook in JupyterLab
+cannot touch the live training run), and artifacts that do not exist yet are reported by the
+``artifact()`` helper as "not available yet (produced by ...)" instead of raising. Scratch files
+go to ``<project>/.notebook_tmp`` and are removed by the last cell.
+
+    .venv/bin/python scripts/build_notebook.py            # write the notebook (no outputs)
+    CUDA_VISIBLE_DEVICES="" .venv/bin/python -m jupyter nbconvert --to notebook --execute \
+        --inplace --ExecutePreprocessor.timeout=600 FinalProject_PhishingDetector.ipynb
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+import nbformat
+from nbformat.v4 import new_code_cell, new_markdown_cell, new_notebook
+
+ROOT = Path(__file__).resolve().parent.parent
+OUT = ROOT / "FinalProject_PhishingDetector.ipynb"
+
+CELLS: list[tuple[str, str]] = []
+
+
+def md(text: str) -> None:
+    CELLS.append(("markdown", text.strip("\n")))
+
+
+def code(text: str) -> None:
+    CELLS.append(("code", text.strip("\n")))
+
+
+# =============================================================================== title
+md("""
+# BITS Phishing Detector: a from-scratch 337M-parameter email classifier
+
+**Nathan King, Build IT Systems LLC (BITS)**
+Skillspire AI & Machine Learning course, final project. Presented in class, late October / early November 2026.
+
+**Abstract.** Small-business inboxes are triaged by people, not security teams, so this project builds a
+*defensive* phishing / spam triage model that a person can paste an email into. It is a text **classifier**
+(three classes: **legitimate**, **phishing**, **spam**) and nothing in it is pretrained or borrowed: the model is a
+24-layer BERT-style encoder written in plain PyTorch (`src/model.py`, 337M parameters), the tokenizer is a
+byte-level BPE trained on our own corpus, and the weights come from two training phases on one RTX 3090 Ti:
+masked-language-model pretraining on deduplicated Enron email, then fine-tuning on public labelled corpora
+with a partial-label loss (the TREC / CEAS positives never separated spam from phishing, so they train as
+"unwanted"). The clean TREC-06 corpus is held out as the binary test set and a separate clean-provenance
+three-way holdout gives the 3x3 confusion matrix. The classifier is exported to safetensors and deployed as a
+Gradio app on a Hugging Face Space (ZeroGPU). This notebook follows the nine steps of the course reference
+notebook and loads the real artifacts of the project; anything that is still being produced while the GPU
+trains is reported as "not available yet" together with the command that produces it.
+""")
+
+# =============================================================================== Step 1
+md("""
+**Step 1: Install Dependencies & Setup Device**
+
+Nothing is installed here: the project runs in a pinned virtual environment (`requirements-train.txt`, every pin
+verified to ship a Python 3.14 wheel). This cell imports the libraries, sets up device-agnostic code, seeds the
+RNGs and prints the runtime facts that drove the VRAM planning. The GPU is currently occupied by the pretraining
+run, so the notebook pins itself to the CPU (`USE_GPU = False` hides the GPU from this kernel before torch is
+imported; the executed copy was additionally run with `CUDA_VISIBLE_DEVICES=""`). Flip the flag only once the
+training queue has finished.
+""")
+code("""
+import os, sys, json, time, inspect, hashlib, subprocess, shutil
+from pathlib import Path
+from timeit import default_timer as timer
+
+# Keep this kernel off the GPU while the pretraining / fine-tuning queue owns it (20+ GiB of 24 GiB in use):
+# the variable must be set before torch is imported anywhere in this process (app/app.py in Step 7 included).
+USE_GPU = False
+if not USE_GPU:
+    os.environ["CUDA_VISIBLE_DEVICES"] = ""
+
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+from IPython.display import display, Image, Markdown
+
+T_NOTEBOOK_START = timer()
+pd.set_option("display.width", 160); pd.set_option("display.max_columns", 40)
+
+# Project root = the directory that holds config.yaml (works from the notebook's folder or a subfolder).
+ROOT = Path.cwd().resolve()
+while not (ROOT / "config.yaml").exists() and ROOT.parent != ROOT:
+    ROOT = ROOT.parent
+sys.path.insert(0, str(ROOT / "src"))
+import common
+CFG = common.load_config(ROOT / "config.yaml")
+PRETRAIN_RUN = "large-24L"          # RUN_NAME in scripts/launch_pretrain.sh
+print("project root:", ROOT)
+
+MISSING: list[tuple[str, str]] = []
+def artifact(rel: str, produced_by: str):
+    \"\"\"Path to a project artifact, or None (with a one-line notice) if it does not exist yet.\"\"\"
+    p = ROOT / rel
+    if p.exists():
+        return p
+    print(f"not available yet — {rel} (produced by: {produced_by})")
+    MISSING.append((rel, produced_by))
+    return None
+
+def show(df, title=None, fmt=None):
+    if title:
+        display(Markdown(f"**{title}**"))
+    display(df.style.format(fmt) if fmt else df)
+""")
+code("""
+import torch
+import torch.nn as nn
+from torch.utils.data import DataLoader
+
+# Setup device-agnostic code (the training scripts do the same; here the GPU is busy pretraining)
+device = "cuda" if torch.cuda.is_available() else "cpu"
+print(f"Using device: {device}  (USE_GPU={USE_GPU}, CUDA_VISIBLE_DEVICES={os.environ.get('CUDA_VISIBLE_DEVICES')!r})")
+
+# Safety net: while a training session owns the GPU this notebook must not see it (a 337M fp32 model + the live
+# pretraining run would exceed 24 GiB and crash the trainer).
+def tmux_session_alive(name: str) -> bool:
+    try:
+        return subprocess.run(["tmux", "has-session", "-t", name], capture_output=True).returncode == 0
+    except FileNotFoundError:
+        return False
+TRAINING_LIVE = [s for s in ("phish-pretrain", "phish-finetune") if tmux_session_alive(s)]
+print("live training tmux sessions:", TRAINING_LIVE or "none")
+assert not (TRAINING_LIVE and device == "cuda"), f"GPU visible while {TRAINING_LIVE} is training: set USE_GPU = False and restart the kernel"
+
+# Set seeds for reproducibility (random, numpy, torch CPU + CUDA)
+common.set_seed(CFG["seed"])
+torch.manual_seed(CFG["seed"])
+if torch.cuda.is_available():
+    torch.cuda.manual_seed(CFG["seed"])
+
+print("\\nRuntime facts (common.print_device_info):")
+DEVICE_INFO = common.print_device_info()
+print(f"\\nCPU threads for torch: {torch.get_num_threads()}")
+""")
+code("""
+# The GPU facts recorded by the smoke test on 2026-10-07 (the GPU is not visible to this kernel).
+smoke = artifact("reports/smoke_test.json", "src/smoke_test.py")
+SMOKE = json.loads(smoke.read_text()) if smoke else {}     # reused by Step 5 and Results (no retyped numbers)
+if smoke:
+    s = SMOKE["device"]
+    print("Training GPU (from reports/smoke_test.json):")
+    for k in ("device", "sm", "vram_gib", "bf16", "torch", "cuda_build", "cudnn", "python"):
+        print(f"{k:>10}: {s[k]}")
+
+print("\\nPinned training environment (requirements-train.txt):")
+print((ROOT / "requirements-train.txt").read_text())
+""")
+
+# =============================================================================== Step 2
+md("""
+**Step 2: Download Data & Create DataLoaders**
+
+The course reference downloads FashionMNIST from `torchvision.datasets`; there is no phishing corpus in torchvision,
+so our data comes from the Hugging Face Hub and is rebuilt locally by `src/data_prep.py` (download, normalise,
+dedupe, decontaminate, split, assert zero leakage). The Hub datasets that were inspected:
+
+| dataset | role | verdict |
+|---|---|---|
+| `puyang2025/seven-phishing-email-datasets` | labelled emails (TREC-05/06/07, CEAS-08, Enron-Spam, SpamAssassin, Ling) | **used**; its own train/test split leaks (5.1% exact / ~40% near duplicates), so we split ourselves |
+| `corbt/enron-emails` | unlabelled Enron mail for MLM pretraining | **used** after removing 52.7% exact duplicates and stripping quoted tails |
+| `it4lia/PhishingEmailCuratedDatasets_Cleaned` | labelled | **excluded**: no body text, only subjects + 53 numeric features |
+| `zefang-liu/phishing-email-dataset` | labelled | **excluded**: 91.9% exact subset of the primary set, lowercased, and 20% of its "safe" rows contain the token *enron* (a shortcut) |
+| `kudzaiprichard/aura-phishing-email-corpus` (Nazario + Nigerian fraud) | phishing-only corpora for the three-way labels | **used** (added in RUN_LOG entry 7) |
+| `locuoco/...-300000` | ready-made 3-class set | **rejected**: 9.9% of its rows are copies of our TREC-06 test set and its "phish" label is a preprocessing artifact |
+
+**Holdouts.** TREC-06 (16,302 rows, 23.9% positive) is the binary test set: it is the one sub-corpus that is not
+Enron-derived (89% of TREC-05 ham and 86% of Enron-Spam ham appear in the pretraining corpus), and it was never seen
+by the tokenizer, the pretraining or the fine-tuning. A second, clean-provenance **three-way holdout** (3,220 rows:
+legitimate / phishing / spam) gives the 3x3 confusion matrix. Train and validation are split by near-duplicate
+cluster so no spam template straddles the two.
+""")
+code("""
+manifest = json.loads((ROOT / "data/processed/splits_manifest.json").read_text())
+ds = manifest["datasets"]
+rows = [
+    ("labelled primary", ds["labelled_primary"]["repo"], ds["labelled_primary"]["rows_loaded"],
+     f"hub splits {ds['labelled_primary']['hub_splits_concatenated']} concatenated (the hub split leaks)"),
+    ("pretraining corpus", ds["pretrain_corpus"]["repo"], ds["pretrain_corpus"]["rows_total"],
+     f"-> {manifest['pretrain']['rows_out']:,} docs after dedup/quote stripping"),
+] + [("excluded", r, "-", reason) for r, reason in zip(ds["excluded"],
+       ("no body text (subjects + 53 numeric features only)", "91.9% exact subset of the primary set, lowercased, 'enron' token shortcut"))]
+rows.append(("phishing corpora (3-way)", CFG["data"]["threeway"]["champa"]["repo"],
+             sum(manifest["threeway"]["new_rows"]["loaded_by_source"].values()), "Nazario + Nigerian_Fraud, CC-BY-4.0"))
+show(pd.DataFrame(rows, columns=["role", "Hub dataset", "rows loaded", "note"]).astype({"rows loaded": "string"}), "Hub datasets")
+print(f"data build: {manifest['timestamp']}  seed {manifest['seed']}  elapsed {manifest['elapsed_seconds']} s  peak RSS {manifest['peak_rss_gb']} GB")
+""")
+code("""
+# Dedup / leakage procedure: every row is normalised (NFKC -> lowercase -> [a-z0-9]) and keyed by sha1 of the full
+# text, a 300-char prefix and subject+body; near-duplicates share >= 3 of their 50-char shingles (stride 25).
+# These are the checks data_prep asserts to be ZERO (recomputed independently from the written files by `check3`).
+leak = manifest["threeway"]["leakage"]
+tbl = pd.DataFrame([{"check": k, "count": v["count"], "asserted zero": v["asserted"], "pass": v["pass"], "detail": v.get("detail", "")}
+                    for k, v in leak.items()])
+show(tbl[tbl["asserted zero"]].reset_index(drop=True), "Leakage checks asserted to be zero (binary gate + three-way)")
+show(tbl[~tbl["asserted zero"]].reset_index(drop=True), "Informational counters (not asserted)")
+D = manifest["drops"]
+drops = pd.DataFrame([
+    ("labelled rows with < 20 alphanumerics (garbage / empty)", D["labelled_min_alnum"]["rows"]),
+    ("Ling corpus excluded (academic mailing list)", sum(D["ling_excluded_by_label"].values())),
+    ("identical body carrying both labels (label noise)", D["pool_label_conflicts"]["rows_dropped"]),
+    ("exact duplicates in the labelled pool", sum(sum(v.values()) for v in D["pool_exact_dups_by_source_label"].values())),
+    ("training rows that are exact/near duplicates of TREC-06 holdout rows", D["decontamination_vs_holdout"]["by_reason"]["total"]),
+    ("pretraining rows: exact duplicates", manifest["pretrain"]["exact_dups_raw"]),
+    ("pretraining rows matching a validation email", manifest["pretrain"]["val_key_removed"]),
+    ("pretraining rows near-duplicating a holdout email", manifest["pretrain"]["holdout_near_dup_removed"]),
+], columns=["removed", "rows"]).set_index("removed")
+show(drops, "Rows removed by the dedup / decontamination procedure", fmt="{:,}")
+""")
+code("""
+import pyarrow.parquet as pq
+
+PROCESSED = Path(CFG["paths"]["data_processed"])
+LABEL3_NAME = {0: "Legitimate", 1: "Phishing", 2: "Spam", -1: "unwanted (unsplit)"}
+SPLITS = ["finetune_train", "finetune_val", "test_holdout", "test_holdout3"]
+frames = []
+for name in SPLITS:
+    t = pq.read_table(PROCESSED / f"{name}.parquet", columns=["source", "label", "label3"]).to_pandas()
+    t["split"] = name
+    frames.append(t)
+lab = pd.concat(frames, ignore_index=True)
+lab["label3"] = lab["label3"].astype(int); lab["label"] = lab["label"].astype(int)
+lab["class"] = lab["label3"].map(LABEL3_NAME)
+
+by_split = pd.crosstab(lab["split"], lab["class"]).reindex(SPLITS)
+by_split["rows"] = by_split.sum(axis=1)
+by_split["binary positive rate"] = lab.groupby("split")["label"].mean().reindex(SPLITS).round(4)
+show(by_split, "Rows per split by label3 (EDA)")
+show(pd.crosstab([lab["split"], lab["source"]], lab["class"]), "Rows per split x source x class")
+""")
+code("""
+fig, ax = plt.subplots(figsize=(9, 3.8))
+cols = ["Legitimate", "Phishing", "Spam", "unwanted (unsplit)"]
+colors = ["#2a78d6", "#eb6834", "#eda100", "#9aa0a6"]
+bottom = np.zeros(len(SPLITS))
+for c, col in zip(cols, colors):
+    vals = by_split[c].reindex(SPLITS).fillna(0).to_numpy() if c in by_split else np.zeros(len(SPLITS))
+    ax.bar(SPLITS, vals, bottom=bottom, label=c, color=col)
+    bottom += vals
+ax.set_ylabel("rows"); ax.set_title("Labelled splits by class", loc="left"); ax.legend(frameon=False)
+for s in ("top", "right"): ax.spines[s].set_visible(False)
+plt.tight_layout(); plt.show()
+""")
+code("""
+# Tokenizer: byte-level BPE, vocab 32,768, NFKC only (case preserved: ALL-CAPS is a spam signal), trained on the
+# pretraining corpus + finetune_train ONLY (val and the holdouts are refused by name; 0 holdout hash collisions).
+from tokenizer import load_tokenizer, encode_batch
+tok_report = json.loads((ROOT / "data/processed/tokenizer_report.json").read_text())
+tr = tok_report["train"]
+print(f"vocab {tr['final_vocab_size']:,}  specials {tr['special_token_ids']}  normalizer {tr['normalizer']}  "
+      f"pre-tokenizer {tr['pre_tokenizer']}\\ntrained on {list(tr['training_files'])} in {tr['train_seconds']} s; "
+      f"excluded {tr['excluded_files']}; holdout exact collisions {tr['holdout_exact_collisions']}")
+show(pd.DataFrame(tok_report["stats"]["splits"]).set_index("split")
+       [["rows", "mean_tokens", "median_tokens", "p95_tokens", "chars_per_token", "pct_truncated", "mean_tokens_by_label"]],
+     "Token-length statistics per split (tokenizer.py stats; max_seq_len 512)")
+
+TOK = load_tokenizer(CFG["paths"]["tokenizer"])
+for ex in tok_report["stats"]["examples"][:3]:
+    enc = TOK.encode(ex["text"])
+    print(f"\\n[{ex['name']}] {ex['text'][:90]!r}\\n  {len(enc.ids)} tokens: {enc.tokens[:28]}{' ...' if len(enc.ids) > 28 else ''}")
+""")
+code("""
+# Token-length histogram per class, tokenised live on a seeded 5,000-row sample of finetune_train.
+rng = np.random.default_rng(CFG["seed"])
+full = pq.read_table(PROCESSED / "finetune_train.parquet", columns=["text", "label3"])
+idx = np.sort(rng.choice(full.num_rows, size=5000, replace=False))
+sample = full.take(idx).to_pandas()
+sample["label3"] = sample["label3"].astype(int)
+sample["n_tokens"] = [len(e.ids) for e in TOK.encode_batch_fast(sample["text"].fillna("").tolist())]
+
+fig, ax = plt.subplots(figsize=(9, 3.8))
+bins = np.logspace(1, 4, 50)
+for c, col in zip([0, 1, 2, -1], ["#2a78d6", "#eb6834", "#eda100", "#9aa0a6"]):
+    v = sample.loc[sample["label3"] == c, "n_tokens"]
+    if len(v):
+        ax.hist(v, bins=bins, histtype="step", linewidth=1.6, color=col, label=f"{LABEL3_NAME[c]} (n={len(v)}, mean {v.mean():.0f})")
+ax.axvline(CFG["model"]["max_seq_len"], color="black", linestyle="--", linewidth=1, label="max_seq_len 512 (truncation)")
+ax.set_xscale("log"); ax.set_xlabel("tokens per email (log)"); ax.set_ylabel("emails")
+ax.set_title("Token length by class, 5k-row sample of finetune_train", loc="left"); ax.legend(frameon=False, fontsize=8)
+for s in ("top", "right"): ax.spines[s].set_visible(False)
+plt.tight_layout(); plt.show()
+print(f"{100 * (sample['n_tokens'] > 512).mean():.1f}% of the sample is longer than 512 tokens and gets truncated. "
+      "Length is a known confound: legitimate mail is ~2x longer on average; the error analysis checks whether the model leans on it.")
+""")
+code("""
+# Build REAL DataLoaders with the training script's own functions (src/finetune.py) on a 512-row seeded sample.
+import finetune
+ft_cfg = CFG["finetune"]
+MULTIPLE = int(ft_cfg["dynamic_padding_multiple"]); MAX_LEN = int(CFG["model"]["max_seq_len"])
+BATCH_SIZE = int(ft_cfg["batch_size"]); NUM_LABELS = int(ft_cfg["num_labels"])
+
+labelled = finetune.read_split(PROCESSED / "finetune_train.parquet", CFG["data"]["holdout_test_source"],
+                               limit=512, seed=CFG["seed"], num_labels=NUM_LABELS)        # refuses the TREC-06 holdout by name AND source
+encoded = finetune.encode_split("train-sample", TOK, labelled, MAX_LEN, MULTIPLE)        # [CLS] text [SEP], truncate 512, store trimmed
+batches = finetune.epoch_batches(encoded.lengths, BATCH_SIZE, seed=CFG["seed"], epoch=0)  # seeded shuffle + length bucketing
+train_dataloader = finetune.make_loader(encoded, batches, MULTIPLE)                      # torch.utils.data.DataLoader
+test_batches = finetune.eval_batches(encoded.lengths, int(ft_cfg["eval_batch_size"]))
+test_dataloader = finetune.make_loader(encoded, test_batches, MULTIPLE)
+
+print(f"{len(labelled)} rows -> {len(batches)} training batches of {BATCH_SIZE} (shuffled, length-bucketed) and "
+      f"{len(test_batches)} eval batches of {ft_cfg['eval_batch_size']} (sorted by length, no shuffle)")
+print("class counts in the sample:", finetune.label_counts(labelled.label3, NUM_LABELS))
+idx, ids, mask, labels, allowed = next(iter(train_dataloader))
+print(f"\\none batch: input_ids {tuple(ids.shape)} {ids.dtype} | attention_mask {tuple(mask.shape)} {mask.dtype} | "
+      f"labels {tuple(labels.shape)} | allowed-class mask {tuple(allowed.shape)}")
+print(f"padding rule: pad to the longest row in the batch ({int(mask.sum(1).max())} tokens) rounded up to a multiple of "
+      f"{MULTIPLE} -> L = {ids.shape[1]}; padded positions carry [PAD]=0 and mask False")
+print("\\nfirst rows of the batch (label_set -> allowed-class mask over [Legitimate, Phishing, Spam]):")
+for r in range(min(6, len(idx))):
+    i = int(idx[r])
+    print(f"  row {i:>4}  source {labelled.sources[i]:<9} label3 {int(labelled.label3[i]):>2}  allowed {allowed[r].int().tolist()}  "
+          f"tokens {int(mask[r].sum()):>3}  text {labelled.texts[i][:60]!r}")
+print("\\n'unwanted' rows (TREC/CEAS positives) allow BOTH phishing and spam: the loss is -log(P(phishing)+P(spam)), the partial-label NLL.")
+""")
+code("""
+# Pretraining data: documents are tokenised once, concatenated with [SEP] after each, and cut into packed windows of
+# [CLS] + 511 stream tokens (no padding, no attention mask -> the Flash SDPA kernel). Dynamic 15% masking per batch.
+pt = json.loads((ROOT / "data/processed/pretrain_tokens.json").read_text())
+print(f"documents {pt['n_docs']:,} ({pt['n_docs_train']:,} train / {pt['n_docs_val']:,} val: {pt['val_rule']})")
+print(f"tokens    {pt['n_tokens_train']:,} train + {pt['n_tokens_val']:,} val  ({pt['tokens_per_char']} tokens/char over {pt['chars']:,} chars; "
+      f"encoded in {pt['encode_seconds']} s, stored as {pt['dtype']})")
+print(f"packing   {pt['packing']}")
+summ = artifact(f"checkpoints/pretrain/{PRETRAIN_RUN}/run_summary.json", f"src/pretrain.py train --run-name {PRETRAIN_RUN}")
+if summ:
+    d = json.loads(summ.read_text())["config"]
+    print(f"windows   {d['data']['n_windows']:,} x 512 per epoch -> {d['steps_per_epoch']} optimizer steps at effective batch "
+          f"{d['effective_batch']} (micro-batch {d['micro_batch']} x {d['grad_accumulation']} accumulation); {d['data']['n_val_windows']:,} val windows")
+""")
+
+# =============================================================================== Step 3
+md("""
+**Step 3: Define the Neural Network**
+
+The reference notebook defines a small CNN; ours is `src/model.py`, a from-scratch **pre-LayerNorm BERT-style
+encoder** in plain PyTorch (no `transformers`, no pretrained weights): token + learned position embeddings, 24
+encoder layers (`x = x + Dropout(OutProj(SDPA(QKV(LN1(x)))))`, `x = x + Dropout(FFN_down(GELU(FFN_up(LN2(x)))))`),
+a final LayerNorm, and two heads. Pre-LN is chosen because a 24-layer stack trained from scratch tolerates the
+learning rate and warmup far better than post-LN. Attention runs through `F.scaled_dot_product_attention` so the fused
+Flash / memory-efficient kernels never materialise the 512x512 score matrix. The **MLM head** ties its decoder to the
+token embedding (saves 33.5M parameters); the **classification head** reads the `[CLS]` position and outputs three
+logits (legitimate / phishing / spam); rows whose corpus never separated spam from phishing train on the probability
+mass of {phishing, spam}. The course requires at least 2 layers; this model has 24.
+""")
+code("""
+import model as phish_model
+from model import EncoderConfig, EncoderLayer, EncoderForMLM, EncoderForClassification, count_parameters
+
+import re
+src_cfg = inspect.getsource(EncoderConfig).splitlines()
+print("class EncoderConfig:   # dataclass: the dials of the encoder")
+print("\\n".join(l for l in src_cfg if re.match(r"^    \\w+: [\\w| ]+ = ", l)))
+print("\\nclass EncoderLayer(nn.Module):   # one pre-LN block; 24 of these are stacked")
+print(inspect.getsource(EncoderLayer.__init__))
+print(inspect.getsource(EncoderLayer.forward))
+""")
+code("""
+# Instantiate both wrappers on the CPU (fp32, ~1.3 GB each) and count parameters.
+model_cfg = EncoderConfig.from_dict({**CFG["model"], "num_labels": CFG["finetune"]["num_labels"]})
+print("config:", model_cfg.to_dict())
+mlm_model = EncoderForMLM(model_cfg)
+cls_model = EncoderForClassification.from_pretrained_encoder(mlm_model)   # encoder weights copied, fresh 3-class head
+counts = pd.DataFrame({"EncoderForMLM (tied head)": count_parameters(mlm_model),
+                       "EncoderForClassification (3 classes)": count_parameters(cls_model)})
+show(counts, "Parameter counts", fmt="{:,}")
+assert mlm_model.head.decoder.weight is mlm_model.encoder.embeddings.tok.weight, "MLM head must be tied"
+print(f"the classifier has {model_cfg.n_layers} encoder layers (course minimum: 2); head: CLS -> dropout -> Linear -> tanh -> dropout -> Linear({model_cfg.num_labels})")
+del mlm_model, cls_model
+""")
+code("""
+# Why 24 layers / d=1024 / vocab 32,768: the parameter-count table from docs/research/research_model-math.md section 1
+# (configs A-F; the brief asked for ~330M parameters). Config A is the BERT-large shape and lands +2.3% over the target.
+math_md = (ROOT / "docs/research/research_model-math.md").read_text().splitlines()
+start = next(i for i, l in enumerate(math_md) if l.startswith("## 1."))
+end = next(i for i, l in enumerate(math_md) if l.startswith("## 2."))
+table = [l for l in math_md[start:end] if l.startswith("|")]
+header = [h.strip() for h in table[0].strip("|").split("|")]
+body = [[c.strip().replace("**", "") for c in l.strip("|").split("|")] for l in table[2:]]
+show(pd.DataFrame(body, columns=header).set_index("config"), "Candidate configurations (research_model-math.md section 1)")
+print(f"Chosen: A = {model_cfg.n_layers}L d{model_cfg.d_model} h{model_cfg.n_heads} f{model_cfg.d_ff} V{model_cfg.vocab_size} -> "
+      f"{counts.loc['total'].iloc[0]:,} (pretraining, tied head) / {counts.loc['total'].iloc[1]:,} (classifier), counted above.")
+""")
+
+# =============================================================================== Step 4
+md("""
+**Step 4: Train the Model**
+
+Training happens in two phases, both far too long for a notebook cell, so they run as scripts inside `tmux` on the
+RTX 3090 Ti and this notebook reads their metrics. **Phase 1, MLM pretraining** (`src/pretrain.py`): AdamW, peak LR
+1.5e-4, 6% warmup, cosine decay to 10%, effective batch 264 sequences x 512 tokens, bf16 autocast, 12 epochs over 107.7M
+tokens (~9,552 optimizer steps). **Phase 2, fine-tuning** (`src/finetune.py`): the instructor's dials are CLI flags
+(`--n-layers`, `--epochs`, `--loss cross_entropy|weighted_cross_entropy|focal`, `--lr`, `--freeze-layers`, `--synthetic`),
+AdamW with linear warmup/decay, **early stopping** on the validation F1 of the unwanted roll-up (patience 2), and every
+run appends one line to `reports/experiments.jsonl` so the experimentation table is generated from evidence. The six
+fine-tuning experiments (loss function x3, synthetic data, 12 layers, scratch init) are queued in
+`scripts/run_experiments.sh` (tmux `phish-finetune`), which waits for `final/` and then trains, evaluates and exports
+each run unattended (RUN_LOG entry 9).
+""")
+code("""
+print("Phase 1 launch (scripts/launch_pretrain.sh; Nathan's settings on 2026-10-08):")
+print("    EPOCHS=12 MICRO_BATCH=24 scripts/launch_pretrain.sh        # -> tmux session 'phish-pretrain'")
+print("    tmux attach -t phish-pretrain                              # Ctrl-b d detaches; Ctrl-C saves a checkpoint and exits")
+print("    RESUME=1 scripts/launch_pretrain.sh                        # continue from the latest checkpoint")
+print("Phase 2 (after final/ exists), the dials are flags:")
+print("    .venv/bin/python src/finetune.py train --run-name ft-ce   --loss cross_entropy --epochs 3 --lr 3e-5")
+print("    .venv/bin/python src/finetune.py train --run-name ft-focal --loss focal --focal-gamma 2.0")
+print("    .venv/bin/python src/finetune.py train --run-name ft-12L   --n-layers 12          # fewer layers")
+print("    .venv/bin/python src/finetune.py list                                              # experiments table\\n")
+queue = ROOT / "scripts/run_experiments.sh"
+if queue.exists():
+    lines = queue.read_text().splitlines()
+    a = next(i for i, l in enumerate(lines) if l.startswith("EXPERIMENTS=("))
+    b = next(i for i in range(a, len(lines)) if lines[i].strip() == ")")
+    print("Queued experiments (scripts/run_experiments.sh, tmux 'phish-finetune'; name | flags, all --init pretrained unless scratch):")
+    print("\\n".join(lines[a:b + 1]), "\\n")
+print((ROOT / "scripts/launch_pretrain.sh").read_text())
+""")
+code("""
+# Phase 1 live metrics: checkpoints/pretrain/<run>/metrics.csv grows while the run trains.
+mpath = artifact(f"checkpoints/pretrain/{PRETRAIN_RUN}/metrics.csv", f"src/pretrain.py train --run-name {PRETRAIN_RUN}")
+final_cfg = artifact(f"checkpoints/pretrain/{PRETRAIN_RUN}/final/config.json",
+                     f"written automatically when the pretraining run completes (or: src/pretrain.py export --run-name {PRETRAIN_RUN})")
+if mpath:
+    # the trainer appends while we read: drop a partially written last row instead of failing on NaN
+    pm = pd.read_csv(mpath).dropna(subset=["step", "loss", "loss_avg", "lr", "tokens_per_s", "train_seconds_total"])
+    last = pm.iloc[-1]
+    plan = json.loads((ROOT / f"checkpoints/pretrain/{PRETRAIN_RUN}/run_summary.json").read_text())["config"]
+    status = "COMPLETE" if final_cfg else "RUN IN PROGRESS"
+    print(f"[{status}] step {int(last.step):,} / {plan['total_steps']:,}  epoch {last.epoch_frac:.2f} / {plan['epochs']}  "
+          f"loss {last.loss:.4f} (avg {last.loss_avg:.4f})  lr {last.lr:.2e}  {int(last.tokens_per_s):,} tokens/s  "
+          f"peak reserved {last.max_reserved_gb} GiB")
+    val = pm.dropna(subset=["val_mlm_loss"])
+    if len(val):
+        v = val.iloc[-1]
+        print(f"last val MLM loss {v.val_mlm_loss:.4f}  val masked-token accuracy {v.val_mlm_acc:.3f} at step {int(v.step)}  (ln V = {np.log(CFG['model']['vocab_size']):.2f})")
+    print(f"Total training time: {last.train_seconds_total:.2f} seconds{' so far' if not final_cfg else ''} "
+          f"({last.train_seconds_total / 3600:.2f} h; projected {last.train_seconds_total / last.step * plan['total_steps'] / 3600:.1f} h for the full plan)")
+
+    fig, ax = plt.subplots(figsize=(9.5, 4))
+    ax.plot(pm.step, pm.loss, color="#9ec3ee", linewidth=0.9, label="train MLM loss (20-step)")
+    ax.plot(pm.step, pm.loss_avg, color="#2a78d6", linewidth=1.8, label="train MLM loss (running avg)")
+    if len(val):
+        ax.plot(val.step, val.val_mlm_loss, "o-", color="#eb6834", label="val MLM loss (every 500 steps)")
+    ax.axhline(np.log(CFG["model"]["vocab_size"]), color="grey", linestyle=":", label="ln(vocab) = uniform guess")
+    ax.set_xlabel("optimizer step"); ax.set_ylabel("MLM loss"); ax.set_title(f"Pretraining {PRETRAIN_RUN}: {status.lower()}", loc="left")
+    ax2 = ax.twinx(); ax2.plot(pm.step, pm.lr, color="#1baf7a", linewidth=1, label="learning rate"); ax2.set_ylabel("lr", color="#1baf7a")
+    h1, l1 = ax.get_legend_handles_labels(); h2, l2 = ax2.get_legend_handles_labels(); ax.legend(h1 + h2, l1 + l2, frameon=False, fontsize=8)
+    for s in ("top",): ax.spines[s].set_visible(False); ax2.spines[s].set_visible(False)
+    plt.tight_layout(); plt.show()
+""")
+code("""
+# reports/pretrain_curves.png is written by `pretrain.py plot --run-name <run>`; show it only if it belongs to this run.
+png = artifact("reports/pretrain_curves.png", f"src/pretrain.py plot --run-name {PRETRAIN_RUN}")
+if png and summ:
+    import re as _re
+    S = json.loads(summ.read_text())
+    # run_summary.json records the finish time and the log path (pretrain-YYYYMMDD-HHMMSS.log); the launch time comes from the latter.
+    started = S.get("started")
+    if not started and S.get("log"):
+        m = _re.search(r"(\d{4})(\d{2})(\d{2})-(\d{2})(\d{2})(\d{2})", Path(S["log"]).name)
+        if m:
+            started = "{}-{}-{}T{}:{}:{}".format(*m.groups())
+    started = started or S.get("finished") or ""
+    if time.strftime("%Y-%m-%dT%H:%M:%S", time.localtime(png.stat().st_mtime)) >= started:
+        display(Image(filename=str(png), width=900))
+    else:
+        print(f"reports/pretrain_curves.png is from an earlier DEV self-test run (older than the {PRETRAIN_RUN} launch at {started}); "
+              f"regenerate with: src/pretrain.py plot --run-name {PRETRAIN_RUN}")
+""")
+code("""
+# Phase 2 experiments table (reports/experiments.jsonl, one JSON line per run). Dev self-test runs (tiny subsets, a
+# handful of steps, scratch init) are marked: they prove the code path, they are not results.
+def is_dev(rec):
+    # A dev self-test trains on a row subset (--limit-rows) or is named dev*; a real run capped with --max-steps
+    # (a legitimate time budget) is NOT a dev run.
+    d = rec.get("dials", {})
+    return bool(rec.get("dev_run") or d.get("limit_rows") or rec["run_name"].startswith("dev"))
+
+exp_path = artifact("reports/experiments.jsonl", "src/finetune.py train ... (one line per run) and src/evaluate.py run")
+EXPERIMENTS = [json.loads(l) for l in exp_path.read_text().splitlines() if l.strip()] if exp_path else []
+if exp_path and not EXPERIMENTS:
+    print("reports/experiments.jsonl exists but is empty: the dev self-test lines were removed (RUN_LOG entry 9); "
+          "the queued runs append to it (produced by: scripts/run_experiments.sh -> src/finetune.py train).")
+if EXPERIMENTS:
+    rows = []
+    for r in EXPERIMENTS:
+        if r.get("stage") == "evaluate":
+            continue
+        d, v = r.get("dials", {}), r.get("val") or {}
+        rows.append({"run": r["run_name"], "dev run?": "DEV" if is_dev(r) else "", "init": r.get("init"), "classes": d.get("num_labels", 2),
+                     "layers": d.get("n_layers"), "frozen": d.get("frozen_layers"), "epochs run/planned": f"{d.get('epochs_run')}/{d.get('epochs_planned')}",
+                     "loss": d.get("loss"), "lr": d.get("lr"), "batch": d.get("batch_size"), "synthetic": d.get("synthetic"),
+                     "train rows": r.get("n_train"), "train min": round(r.get("train_seconds", 0) / 60, 1), "peak GiB": r.get("peak_vram_gb"),
+                     "val acc": v.get("accuracy"), "val P": v.get("precision"), "val R": v.get("recall"), "val F1": v.get("f1"),
+                     "val ROC-AUC": v.get("roc_auc"), "val macro-F1 3w": v.get("macro_f1_3way"), "stopped by": r.get("stopped_by")})
+    EXP_DF = pd.DataFrame(rows).round(4)
+    if len(EXP_DF):
+        show(EXP_DF.set_index("run"), "Fine-tuning experiments (finetune.py list)")
+    real = [r for r in EXPERIMENTS if r.get("stage") != "evaluate" and not is_dev(r)]
+    if not real:
+        print("Only DEV self-test runs exist so far: the real fine-tuning runs start once checkpoints/pretrain/large-24L/final/ exists "
+              "(produced by: scripts/run_experiments.sh -> src/finetune.py train --run-name <run> ...).")
+""")
+code("""
+# Fine-tuning curves of the best real run (highest val F1 of the unwanted roll-up); until one exists, the newest dev run is
+# shown and labelled as such. Early stopping: evaluation every 500 steps, stop after `patience` evaluations without a
+# better val F1; LR: linear warmup over 6% of the steps, then linear decay to 0 (BERT fine-tuning schedule).
+real = [r for r in EXPERIMENTS if r.get("stage") != "evaluate" and not is_dev(r) and (r.get("val") or {}).get("f1") is not None]
+dev = [r for r in EXPERIMENTS if r.get("stage") != "evaluate" and is_dev(r)]
+best = max(real, key=lambda r: r["val"]["f1"]) if real else None
+tag = ""
+if best is None:
+    artifact("checkpoints/finetune/<best run>/metrics.csv", "src/finetune.py train --run-name <run> (after pretraining completes)")
+    if dev:
+        best, tag = dev[-1], " (DEV self-test run, not a result)"
+BEST_FT_RUN = best["run_name"] if best else None
+if best:
+    run_dir = ROOT / "checkpoints/finetune" / best["run_name"]
+    m = pd.read_csv(run_dir / "metrics.csv") if (run_dir / "metrics.csv").exists() else None
+    e = pd.read_csv(run_dir / "epochs.csv") if (run_dir / "epochs.csv").exists() else None
+    print(f"run {best['run_name']}{tag}: init {best['init']}, loss {best['dials']['loss']}, lr {best['dials']['lr']}, "
+          f"{best['steps_run']} steps, best step {best['best_step']}, val F1 {best['val']['f1']:.4f}, stopped by {best['stopped_by']}")
+    if m is not None and len(m):
+        fig, axes = plt.subplots(1, 2, figsize=(11, 3.6))
+        axes[0].plot(m.step, m.loss, color="#2a78d6", label="train loss"); axes[0].set_xlabel("step"); axes[0].set_title(f"{best['run_name']}{tag}: train loss", loc="left", fontsize=10)
+        if e is not None and len(e):
+            axes[1].plot(e.step, e.f1, "o-", color="#eb6834", label="val F1 (unwanted roll-up)")
+            axes[1].plot(e.step, e.roc_auc, "s--", color="#1baf7a", label="val ROC-AUC")
+            best_rows = e[e.is_best == 1]
+            axes[1].scatter(best_rows.step, best_rows.f1, s=90, facecolors="none", edgecolors="black", label="best (checkpoint kept)")
+        axes[1].set_xlabel("step"); axes[1].set_title("validation metrics at each evaluation", loc="left", fontsize=10); axes[1].legend(frameon=False, fontsize=8)
+        for ax in axes:
+            for s in ("top", "right"): ax.spines[s].set_visible(False)
+        plt.tight_layout(); plt.show()
+    # The queue writes reports/finetune_curves_<run>.png per run and reports/finetune_curves.png for the main run; a PNG
+    # older than the run's own experiments.jsonl line is a leftover from an earlier run and is not shown.
+    curves = artifact(f"reports/finetune_curves_{best['run_name']}.png", f"src/finetune.py plot --run-name {best['run_name']} --out reports/finetune_curves_{best['run_name']}.png")
+    curves = curves or (ROOT / "reports/finetune_curves.png" if (ROOT / "reports/finetune_curves.png").exists() else None)
+    if curves and not tag:
+        png_time = time.strftime("%Y-%m-%dT%H:%M:%S", time.localtime(curves.stat().st_mtime))
+        if png_time >= best.get("timestamp", ""):
+            display(Image(filename=str(curves), width=900))
+        else:
+            print(f"{curves.relative_to(ROOT)} predates run {best['run_name']} ({png_time} < {best.get('timestamp')}): stale, not shown; "
+                  f"regenerate with src/finetune.py plot --run-name {best['run_name']}")
+""")
+code("""
+# Loss-function comparison (cross_entropy vs weighted_cross_entropy vs focal) over the REAL runs, same dials otherwise.
+real = [r for r in EXPERIMENTS if r.get("stage") != "evaluate" and not is_dev(r) and r.get("val")]
+if real:
+    cmp = pd.DataFrame([{"run": r["run_name"], "loss": r["dials"]["loss"], "focal_gamma": r["dials"].get("focal_gamma"),
+                         "class_weights": r["dials"].get("class_weights"), "val F1": r["val"]["f1"], "val P": r["val"]["precision"],
+                         "val R": r["val"]["recall"], "val ROC-AUC": r["val"]["roc_auc"], "val PR-AUC": r["val"]["pr_auc"],
+                         "val macro-F1 3w": r["val"].get("macro_f1_3way")} for r in real]).sort_values("val F1", ascending=False)
+    show(cmp.set_index("run"), "Loss-function comparison (real runs)")
+else:
+    artifact("reports/experiments.jsonl (real runs with --loss cross_entropy / weighted_cross_entropy / focal)",
+             "scripts/run_experiments.sh -> src/finetune.py train --run-name ft-ce|ft-weighted|ft-focal --loss <loss>")
+    dev_losses = sorted({(r["dials"].get("loss"), r.get("steps_run"), r.get("n_train")) for r in EXPERIMENTS if r.get("stage") != "evaluate" and is_dev(r)})
+    if dev_losses:
+        print("The DEV lines above (" + "; ".join(f"{l}: {s} steps on {n:,} rows" for l, s, n in dev_losses) + ") only prove the loss paths run.")
+    print("The three loss paths share one code path (src/finetune.py); focal(gamma=0) was verified to equal cross_entropy exactly (RUN_LOG entry 4).")
+""")
+
+# =============================================================================== Step 5
+md("""
+**Step 5: Test Inference Speed and Save the Model**
+
+Exactly like the reference `predict_and_time`: for each of 100 real emails from the TREC-06 holdout, start the timer,
+tokenize, run the forward pass, softmax, stop the timer. The classifier is loaded from the export directory
+(`model_export/<run>/`) when it exists, otherwise from the newest fine-tune `best/` checkpoint, on the **CPU in fp32**
+(the GPU is pretraining). The instructor's benchmark is 0.074 s per sample; the smoke test measured 10.7 ms on the
+3090 Ti in bf16 and ~0.7 s on this CPU at 512 tokens, which is why the Space runs on ZeroGPU. The model is then saved
+as **safetensors** (the format the Space loads; `torch.save` of the same `state_dict` is shown to be equivalent).
+""")
+code("""
+from safetensors.torch import load_file, save_file
+
+def locate_classifier():
+    \"\"\"(directory, kind): the newest export, else the newest real fine-tune best/, else a dev self-test best/.\"\"\"
+    exports = sorted((ROOT / "model_export").glob("*/model.safetensors"), key=lambda p: p.stat().st_mtime) if (ROOT / "model_export").exists() else []
+    if exports:
+        return exports[-1].parent, "export"
+    artifact("model_export/<run>/model.safetensors", "scripts/run_experiments.sh -> src/export.py run --checkpoint checkpoints/finetune/<run>/best")
+    bests = sorted((ROOT / "checkpoints/finetune").glob("*/best/model.safetensors"), key=lambda p: p.stat().st_mtime)
+    real = [b for b in bests if not b.parent.parent.name.startswith("dev")]
+    if real:
+        return real[-1].parent, "fine-tune best checkpoint"
+    if bests:
+        return bests[-1].parent, "dev"
+    artifact("checkpoints/finetune/<run>/best/model.safetensors", "scripts/run_experiments.sh -> src/finetune.py train --run-name <run>")
+    return None, None
+
+CLS_DIR, CLS_KIND = locate_classifier()
+classifier = None
+if CLS_DIR:
+    cj = json.loads((CLS_DIR / "config.json").read_text())
+    if CLS_KIND == "dev":
+        # Describe the self-test from its own metadata (best/config.json 'finetune' = the CLI args, 'init'), never from a literal.
+        ft = cj.get("finetune", {})
+        CLS_KIND = (f"DEV self-test checkpoint ({cj.get('init', '?')} init, {ft.get('max_steps', '?')} steps on "
+                    f"{ft.get('limit_rows', '?')} rows: timing is representative, predictions are not)")
+    cls_cfg = EncoderConfig.from_dict(cj["model"])
+    classifier = EncoderForClassification(cls_cfg)
+    missing_unexpected = classifier.load_state_dict(load_file(str(CLS_DIR / "model.safetensors"), device="cpu"), strict=True)
+    classifier = classifier.float().to("cpu").eval()
+    CLS_LABELS = [cj.get("labels", {}).get(str(i), {0: "Legitimate", 1: "Phishing", 2: "Spam"}.get(i, f"class {i}")) for i in range(cls_cfg.num_labels)]
+    if cls_cfg.num_labels == 2:
+        CLS_LABELS = ["Legitimate", "Spam/Phishing"]
+    print(f"loaded {CLS_KIND}: {CLS_DIR.relative_to(ROOT)}  ({count_parameters(classifier)['total']:,} params, {cls_cfg.n_layers} layers, "
+          f"{cls_cfg.num_labels} classes {CLS_LABELS}, fp32 on cpu)")
+""")
+code("""
+def predict_and_time(model: nn.Module, texts: list[str], device: str = "cpu", max_len: int = 512):
+    \"\"\"Tokenize + forward + softmax per email, timed with timeit.default_timer (the instructor's predict_and_time).\"\"\"
+    model.to(device); model.eval()
+    stats = []
+    with torch.inference_mode():
+        for text in texts:
+            start_time = timer()
+            ids, mask = encode_batch(TOK, [text], max_len, pad_to_multiple_of=MULTIPLE)
+            probs = torch.softmax(model(ids.to(device), mask.to(device)).float(), dim=1)[0]
+            stats.append({"time": timer() - start_time, "tokens": int(mask.sum()), "probs": probs.tolist()})
+    return stats
+
+if classifier is not None:
+    holdout = pq.read_table(PROCESSED / "test_holdout.parquet", columns=["text", "label"])   # read-only, 100 rows, timing only
+    pick = np.random.default_rng(CFG["seed"]).choice(holdout.num_rows, size=100, replace=False)
+    sample_texts = [holdout.column("text")[int(i)].as_py() or "" for i in pick]
+    predict_and_time(classifier, sample_texts[:3])                                             # warm-up (oneDNN kernels), untimed
+    sample_stats = predict_and_time(classifier, sample_texts)
+    times = np.array([s["time"] for s in sample_stats]); toks = np.array([s["tokens"] for s in sample_stats])
+    print(f"Average prediction time per sample: {times.mean():.5f} seconds  (median {np.median(times):.4f}, max {times.max():.4f}; "
+          f"{len(times)} emails, mean {toks.mean():.0f} tokens after truncation to {MAX_LEN}; CPU fp32, {torch.get_num_threads()} threads)")
+    BENCH_S = SMOKE["latency"]["instructor_benchmark_ms"] / 1000 if SMOKE else 0.074
+    msg = f"Instructor benchmark: {BENCH_S:.5f} seconds -> CPU is {times.mean() / BENCH_S:.1f}x slower; the Space therefore runs on ZeroGPU"
+    if SMOKE:
+        msg += f", where the smoke test measured {SMOKE['latency']['gpu_bf16_512_no_mask']['avg_ms'] / 1000:.4f} s (bf16, 512 tokens)"
+    print(msg + ".")
+""")
+code("""
+# Compare with the recorded latencies: reports/latency.json (evaluate.py, GPU bf16 + CPU fp32) and the smoke test.
+rows = []
+lat = artifact("reports/latency.json", f"src/evaluate.py run --checkpoint checkpoints/finetune/<run>/best --split test,test3")
+if lat:
+    L = json.loads(lat.read_text())
+    for k in ("gpu_bf16", "cpu_fp32"):
+        if L.get(k):
+            rows.append({"source": f"evaluate.py {k}", "avg s/sample": L[k]["avg_ms"] / 1000, "p50 s": L[k].get("p50_ms", np.nan) / 1000, "n": L[k].get("n")})
+if SMOKE:
+    SL = SMOKE["latency"]
+    for k in ("gpu_bf16_512_no_mask", "gpu_bf16_512_bool_mask", "cpu_fp32_512", "cpu_fp32_128"):
+        rows.append({"source": f"smoke_test {k}", "avg s/sample": SL[k]["avg_ms"] / 1000, "p50 s": SL[k]["p50_ms"] / 1000, "n": SL[k]["n"]})
+if classifier is not None:
+    rows.append({"source": "this notebook, CPU fp32 (100 holdout emails)", "avg s/sample": float(times.mean()), "p50 s": float(np.median(times)), "n": len(times)})
+rows.append({"source": "instructor benchmark", "avg s/sample": SMOKE["latency"]["instructor_benchmark_ms"] / 1000 if SMOKE else 0.074, "p50 s": np.nan, "n": None})
+show(pd.DataFrame(rows).set_index("source"), "Single-email prediction time (tokenize + forward + softmax)", fmt={"avg s/sample": "{:.4f}", "p50 s": "{:.4f}"})
+""")
+code("""
+# Save the trained model: safetensors (what the Space loads) and torch.save of the same state_dict (the instructor's way).
+NOTEBOOK_TMP = ROOT / ".notebook_tmp"          # project-local scratch (not /tmp, which is RAM-backed here); removed by the last cell
+if classifier is not None:
+    save_dir = NOTEBOOK_TMP / "save"
+    shutil.rmtree(save_dir, ignore_errors=True); save_dir.mkdir(parents=True)
+    sd = {k: v.contiguous() for k, v in classifier.state_dict().items()}
+    SAFE_PATH, PTH_PATH = save_dir / "phishing_classifier.safetensors", save_dir / "phishing_classifier.pth"
+    t0 = timer(); save_file(sd, str(SAFE_PATH), metadata={"format": "pt", "architecture": "EncoderForClassification"}); t_safe = timer() - t0
+    t0 = timer(); torch.save(obj=sd, f=PTH_PATH); t_pth = timer() - t0
+    print(f"Model saved to: {SAFE_PATH}  ({SAFE_PATH.stat().st_size / 2**20:,.0f} MiB, {t_safe:.1f} s)")
+    print(f"Model saved to: {PTH_PATH}  ({PTH_PATH.stat().st_size / 2**20:,.0f} MiB, {t_pth:.1f} s)")
+    back = load_file(str(SAFE_PATH)); pth = torch.load(PTH_PATH, map_location="cpu")
+    same = back.keys() == pth.keys() and all(torch.equal(back[k], pth[k]) for k in back)
+    print(f"safetensors and torch.save round-trip to identical tensors: {same}  ({len(back)} tensors). "
+          f"The real export (src/export.py) writes bf16 safetensors (~675 MB) + config.json + tokenizer.json + model card.")
+    shutil.rmtree(save_dir, ignore_errors=True)
+""")
+
+# =============================================================================== Step 6
+md("""
+**Step 6: Create the App Files (Optional)**
+
+Optional in the reference, not here: the deployed Space is one of the project's deliverables. The reference writes
+`app.py`, `model.py` and `requirements.txt` into a deployment folder. Ours live in `app/`:
+`app.py` (the Gradio interface, a `gr.Interface` with a `gr.Label` and the timed `gr.Number("Prediction Time (s)")`
+plus token-saliency highlights and rule-based cues), `requirements.txt` pinned for the ZeroGPU runtime (torch 2.13.0,
+Python 3.12.12: the highest versions ZeroGPU allows), `README.md` with the Space front matter, and `model.py`, which
+`src/export.py` copies **byte-for-byte** from `src/model.py` next to the exported weights.
+""")
+code("""
+APP_DIR = Path(CFG["paths"]["app_dir"])
+print(f"{APP_DIR.relative_to(ROOT)}/")
+for p in sorted(APP_DIR.iterdir()):
+    if p.name == "__pycache__":
+        continue
+    print(f"  {p.name:<20} {p.stat().st_size:>12,} bytes")
+print("\\n--- app/app.py (first 60 lines) ---")
+print("\\n".join((APP_DIR / "app.py").read_text().splitlines()[:60]))
+""")
+code("""
+print("--- app/requirements.txt ---"); print((APP_DIR / "requirements.txt").read_text())
+readme = (APP_DIR / "README.md").read_text().split("---")
+print("--- app/README.md front matter (Space configuration) ---"); print(readme[1].strip())
+""")
+code("""
+# model.py in the Space must be the byte-identical file the model was trained with (export.py asserts the sha1).
+src_sha = hashlib.sha1((ROOT / "src/model.py").read_bytes()).hexdigest()
+print(f"src/model.py  sha1 {src_sha}")
+app_model = artifact("app/model.py", "src/export.py run --checkpoint checkpoints/finetune/<run>/best  (copies src/model.py byte-for-byte)")
+if app_model:
+    app_sha = hashlib.sha1(app_model.read_bytes()).hexdigest()
+    print(f"app/model.py  sha1 {app_sha}  identical: {app_sha == src_sha}")
+for f in ("model.safetensors", "config.json", "tokenizer.json"):
+    artifact(f"app/{f}", "src/export.py run (copied next to app.py so the Space can also load weights locally)")
+""")
+
+# =============================================================================== Step 7
+md("""
+**Step 7: Launch the Gradio Interface**
+
+The reference launches the demo with `share=True` to get a public link; `gradio.live` links expire after 72 hours, so the
+permanent link for the class is the Hugging Face Space (Step 9). Here the real `app/app.py` is imported and its `predict`
+function (the function the Gradio `Interface` calls) is run on two of the app's built-in **synthetic** example emails,
+printing the three class probabilities and the prediction time. `demo.launch()` is behind a flag that is off by default
+so the notebook executes unattended; flip `LAUNCH_GRADIO = True` to open the interface locally.
+""")
+code("""
+import importlib.util
+
+def stage_model_dir():
+    \"\"\"Directory with model.safetensors + config.json + tokenizer.json for app.py (export dir, or a staged dev checkpoint).\"\"\"
+    exports = sorted((ROOT / "model_export").glob("*/model.safetensors")) if (ROOT / "model_export").exists() else []
+    if exports:
+        return exports[-1].parent
+    if CLS_DIR is None:
+        return None
+    stage = NOTEBOOK_TMP / "stage"          # project-local; symlinks only; removed by the last cell
+    stage.mkdir(parents=True, exist_ok=True)
+    for name, src in (("model.safetensors", CLS_DIR / "model.safetensors"), ("config.json", CLS_DIR / "config.json"),
+                      ("tokenizer.json", Path(CFG["paths"]["tokenizer"]))):
+        dst = stage / name
+        if dst.is_symlink() or dst.exists():
+            dst.unlink()
+        dst.symlink_to(src)
+    return stage
+
+app = None
+model_dir = stage_model_dir()
+if model_dir is None:
+    print("not available yet — app needs a classifier (produced by: src/finetune.py train, then src/export.py run)")
+else:
+    os.environ["PHISH_MODEL_DIR"] = str(model_dir)        # app.py loads from here; otherwise it would download from the private Hub repo
+    print(f"PHISH_MODEL_DIR={model_dir}  ({CLS_KIND})")
+    spec = importlib.util.spec_from_file_location("phish_app", APP_DIR / "app.py")
+    app = importlib.util.module_from_spec(spec); spec.loader.exec_module(app)   # builds `app.demo` (gr.Interface) without launching
+    print(f"\\napp loaded: {app.N_PARAMS:,} params on {app.DEVICE} ({next(app.MODEL.parameters()).dtype}), classes {app.LABELS}, max_len {app.MAX_LEN}")
+    for subject, body in (app.EXAMPLES[0], app.EXAMPLES[4]):
+        label_probs, secs, highlights, signals = app.predict(subject, body)
+        words = [t for t, l in highlights if l]
+        print(f"\\n=== {subject}\\n  probabilities: " + ", ".join(f"{k} {100 * v:.1f}%" for k, v in label_probs.items())
+              + f"\\n  Prediction Time (s): {secs}   verdict line: {signals.splitlines()[0]}\\n  tokens weighed most: {words[:8]}")
+""")
+code("""
+LAUNCH_GRADIO = False    # share links (gradio.live) expire after 72 h; the permanent link is the Space in Step 9
+if LAUNCH_GRADIO and app is not None:
+    app.demo.launch(share=False)
+else:
+    print("Gradio not launched here (LAUNCH_GRADIO = False). Locally: PHISH_MODEL_DIR=model_export/<run> .venv/bin/python app/app.py")
+""")
+
+# =============================================================================== Step 8
+md("""
+**Step 8: Authenticate with Hugging Face**
+
+The reference calls `notebook_login()` and pastes a write token into a widget. We never put a token in a notebook: the
+workstation is logged in once with `hf auth login` (huggingface_hub 1.33.0's CLI), the token lives in the CLI cache, and
+every script reads it from there or from the `HF_TOKEN` environment variable without printing it. This cell only asks the
+Hub who is logged in; it degrades to "offline" without network access.
+""")
+code("""
+import logging; logging.getLogger("httpx").setLevel(logging.WARNING)
+try:
+    from huggingface_hub import HfApi
+    print("Logged in to Hugging Face as:", HfApi().whoami()["name"])
+except Exception as e:   # no network, or not logged in: the notebook still runs
+    print(f"offline / not logged in ({type(e).__name__}); run `hf auth login` once on this machine.")
+print("The token is never stored in this notebook or in the repository (.gitignore excludes *.token and .env).")
+""")
+
+# =============================================================================== Step 9
+md("""
+**Step 9: Upload the App to Hugging Face Spaces**
+
+The reference uploads a folder with `HfApi().upload_folder`. Ours is `scripts/upload_to_hf.py`: it creates the **private**
+model repo and the **private** ZeroGPU Space named in `config.yaml`, uploads `model_export/<run>/` (bf16 safetensors, config,
+tokenizer, model card) to the model repo and `app/` to the Space, sets the `HF_TOKEN` read secret and `PHISH_MODEL_REPO`
+variable so the Space can download the private weights at startup, and prints both URLs. It is a **dry run by default** and
+refuses to write anything without `--yes`; the live upload is run by Nathan from the terminal, never from the notebook.
+Both repos start private and are switched to public for the presentation in the Hub settings.
+""")
+code("""
+hf = CFG["hf"]
+print(f"model repo : https://huggingface.co/{hf['model_repo']}            (private: {hf['private']})")
+print(f"Space      : https://huggingface.co/spaces/{hf['space_repo']}      (sdk {hf['space_sdk']} {hf['space_sdk_version']}, "
+      f"hardware {hf['space_hardware']}, python {hf['space_python_version']}, torch {hf['space_torch']})\\n")
+cmd = [sys.executable, str(ROOT / "scripts/upload_to_hf.py"), "--dry-run"]
+print("$", " ".join(Path(c).name if c.startswith(str(ROOT)) else c for c in cmd))
+res = subprocess.run(cmd, cwd=ROOT, capture_output=True, text=True, timeout=180, env={**os.environ, "CUDA_VISIBLE_DEVICES": ""})
+print((res.stdout + res.stderr).strip()[-4000:])
+print("\\nLive run (Nathan, after approval):  HF_TOKEN_FOR_SPACE=<read token> .venv/bin/python scripts/upload_to_hf.py --model-dir model_export/<run> --yes")
+""")
+
+# =============================================================================== Results
+md("""
+## Results
+
+Performance on the **unseen** test sets, written by `src/evaluate.py` (the only script allowed to open the holdouts).
+The headline is the binary roll-up on TREC-06: P(unwanted) = P(phishing) + P(spam) at threshold 0.5, with recall on the
+unwanted class first (a missed phish is the costly error). The three-way holdout gives the 3x3 confusion matrix; the
+phishing-vs-spam boundary follows corpus provenance and is expected to be softer than legitimate-vs-unwanted.
+""")
+code("""
+def metric_tables(path, title):
+    M = json.loads(path.read_text())
+    at, tf = M.get("at_threshold", {}), M.get("threshold_free", {})
+    print(f"{title}: checkpoint {M.get('checkpoint')}, {M['split_sizes']['rows']:,} rows, positive rate {M['split_sizes']['positive_rate']}, threshold {M.get('threshold')}")
+    binary = {"recall (unwanted)": at.get("recall"), "precision": at.get("precision"), "F1": at.get("f1"), "accuracy": at.get("accuracy"),
+              "specificity": at.get("specificity"), "ROC-AUC": tf.get("roc_auc"), "PR-AUC": tf.get("pr_auc"),
+              "recall @ 1% FPR": (tf.get("recall_at_fpr_1pct") or {}).get("recall"), "recall @ 0.1% FPR": (tf.get("recall_at_fpr_0_1pct") or {}).get("recall")}
+    show(pd.DataFrame(binary, index=["value"]).T, f"{title}: binary roll-up", fmt="{:.4f}")
+    cm = at.get("confusion_matrix")
+    if cm:
+        show(pd.DataFrame([[cm["tn"], cm["fp"]], [cm["fn"], cm["tp"]]], index=["true legitimate", "true unwanted"],
+                          columns=["pred legitimate", "pred unwanted"]), "Binary confusion matrix")
+    lure = (M.get("credential_lure_slice") or {}).get("metrics")
+    if lure:
+        show(pd.DataFrame({k: lure.get(k) for k in ("n", "n_positive", "recall", "precision", "f1", "accuracy")}, index=["credential-lure slice"]).T, "Credential-lure slice")
+    tw = M.get("threeway")
+    if tw and tw.get("confusion_matrix"):
+        labs = tw["confusion_matrix"]["labels"]
+        show(pd.DataFrame(tw["confusion_matrix"]["matrix"], index=[f"true {l}" for l in labs], columns=[f"pred {l}" for l in labs]), "3x3 confusion matrix")
+        show(pd.DataFrame(tw["per_class"]).T, f"Per-class metrics (macro-F1 {tw.get('macro_f1'):.4f})" if tw.get("macro_f1") is not None else "Per-class metrics")
+    return M
+
+RESULTS = {}
+for rel, title in (("reports/metrics.json", "TREC-06 binary holdout"), ("reports/metrics_test3.json", "Three-way holdout")):
+    p = artifact(rel, "src/evaluate.py run --checkpoint checkpoints/finetune/ft-focal/best --split test,test3  (main run ft-focal, selected on validation F1)")
+    if p:
+        RESULTS[rel] = metric_tables(p, title)
+""")
+code("""
+# Ablations on the SAME holdouts: run_experiments.sh evaluates every queued run into reports/eval_<run>/ (one directory per
+# run, so no report overwrites another); the main run (ft-focal: best validation F1 of the unwanted roll-up and best validation
+# three-way macro-F1 among the six, chosen before looking at the test numbers) is also written to the top-level reports/ shown above.
+abl = []
+for d in sorted((ROOT / "reports").glob("eval_*")):
+    mj, m3 = d / "metrics.json", d / "metrics_test3.json"
+    if mj.exists():
+        M = json.loads(mj.read_text()); at, tf = M.get("at_threshold", {}), M.get("threshold_free", {})
+        row = {"run": d.name[len("eval_"):], "TREC-06 recall": at.get("recall"), "precision": at.get("precision"), "F1": at.get("f1"),
+               "ROC-AUC": tf.get("roc_auc"), "PR-AUC": tf.get("pr_auc"), "recall @ 1% FPR": (tf.get("recall_at_fpr_1pct") or {}).get("recall")}
+        if m3.exists():
+            tw = json.loads(m3.read_text()).get("threeway") or {}
+            row["3-way macro-F1"] = tw.get("macro_f1"); row["3-way accuracy"] = tw.get("accuracy")
+        abl.append(row)
+if abl:
+    show(pd.DataFrame(abl).set_index("run").sort_values("F1", ascending=False), "Ablations on the unseen holdouts (reports/eval_<run>/)", fmt="{:.4f}")
+else:
+    artifact("reports/eval_<run>/metrics.json", "scripts/run_experiments.sh -> src/evaluate.py run --out-dir reports/eval_<run> (one per queued run)")
+""")
+code("""
+for rel in ("reports/confusion_matrix.png", "reports/roc_curve.png", "reports/pr_curve.png", "reports/confusion_matrix3_test3.png",
+            "reports/confusion_matrix_test3.png"):
+    p = artifact(rel, "src/evaluate.py run --split test,test3")
+    if p and p.stat().st_size < 2_000_000:
+        display(Image(filename=str(p), width=520))
+for rel in ("reports/error_analysis.md", "reports/error_analysis_test3.md"):
+    p = artifact(rel, "src/evaluate.py run --split test,test3")
+    if p:
+        display(Markdown("\\n".join(p.read_text().splitlines()[:40]) + "\\n\\n*(truncated; full file: " + rel + ")*"))
+""")
+code("""
+# Latency as recorded by evaluate.py (GPU bf16 and CPU fp32 on the same 100 holdout emails) next to the benchmark.
+if lat:
+    L = json.loads(lat.read_text())
+    show(pd.DataFrame({k: v for k, v in L.items() if isinstance(v, dict)}).T, f"reports/latency.json (benchmark {L.get('instructor_benchmark_ms')} ms)")
+else:
+    SL = SMOKE.get("latency", {})
+    g, c = SL.get("gpu_bf16_512_no_mask", {}).get("avg_ms"), SL.get("cpu_fp32_512", {}).get("avg_ms")
+    print("latency.json not written yet (produced by: scripts/run_experiments.sh -> src/evaluate.py run); the measured numbers so far are in the "
+          f"Step 5 table (smoke test GPU {g} ms bf16, CPU {c} ms fp32 at 512 tokens).")
+""")
+
+# =============================================================================== Challenges
+md("""
+## Challenges and what we learned
+
+Distilled from `RUN_LOG.md` (entries 0-9), which records every decision, number and problem.
+
+- **Python 3.14 wheels.** The workstation runs Python 3.14.7; every pin (torch 2.14.1 cu130, tokenizers, safetensors, pyarrow, ...) had to be checked live for a cp314 wheel before the venv was built. The plain PyPI torch wheel *is* the CUDA 13.0 build, and driver 615 (CUDA 13.4) runs any 13.x build by backward compatibility; `uv --torch-backend=auto` would nevertheless have resolved an unverified cu132 build instead of the verified cu130 wheel (uv 0.12.5 iterates cu132 before cu130 for drivers >= 580), so the install pins `--torch-backend=cu130`.
+- **ZeroGPU pins the Space.** ZeroGPU supports only torch 2.8-2.13 on Python 3.12.12, no `torch.compile`, and cannot preload a private model repo, so the Space uses torch 2.13.0 and downloads the weights at startup with an `HF_TOKEN` secret. Training (torch 2.14.1 / 3.14) and serving (2.13.0 / 3.12) are different stacks; safetensors makes that safe.
+- **Datasets without body text.** `it4lia/...Cleaned` turned out to hold only subjects and 53 numeric features; `zefang-liu/...` was a lossy 92% copy of the primary set with an "enron" shortcut. Both were excluded before downloading anything.
+- **Hub-split leakage.** The primary set's own train/test split has 5.1% exact and ~40% near-duplicate test rows in train; we built our own split and asserted zero leakage with independent recomputation.
+- **Enron contamination.** 89% of TREC-05 ham and 86% of Enron-Spam ham are in the pretraining corpus, so neither could be the test set; TREC-06 (5 rows mention Enron) became the holdout, and 1,652 training rows that near-duplicate TREC-06 spam templates were removed.
+- **Near-duplicate decontamination.** Exact-key dedup missed quoted/forwarded 419-letter variants of holdout emails in the MLM corpus; the reviewer caught it, and 50-char shingle matching (>= 3 shared) now gates every split. 52.7% of `corbt/enron-emails` was exact duplicates.
+- **VRAM finder bug.** The first batch finder measured peak memory with one forward/backward and under-reported the real gradient-accumulation peak by one fp32 gradient copy (1.26 GiB); fixed before the long run. Checkpointing turned out unnecessary (B=16 fits with 8 GiB headroom) and costs ~25% throughput.
+- **Effective batch 264, not 256.** Micro-batch 24 does not divide 256; floor division would have reported 256 while training at 240. The trainer now uses ceil(256/24) = 11 accumulation steps and records the realised 264 everywhere, including the LR schedule.
+- **CPU latency reality.** The research estimate of ~0.34 s per 512-token email on the 5950X was 2x optimistic: measured 0.70 s (0.32 s at 128 tokens) because CPU inference is per-op-overhead bound. The 0.074 s benchmark is met on GPU (10.7 ms), hence ZeroGPU rather than a free CPU Space.
+- **The 300k three-class set was rejected.** Inspection showed it is a union of Kaggle re-packagings of corpora we already hold, 9.9% of it copies of our frozen TREC-06 test set, with a "phish" label that agrees with real provenance only 60% of the time and a stopword-stripping artifact that would be a near-perfect label proxy. Not one row was ingested.
+- **Three-way label scarcity.** No public corpus separates spam from phishing at scale: only 3,150 clean phishing training rows (Nazario, Nigerian fraud) plus 1,000 synthetic lures carry the distinction, while 62,805 TREC/CEAS positives are trained as "unwanted" with a partial-label loss. The binary roll-up on TREC-06 is the headline; the three-way split is the weaker decision and is reported as such.
+- **Reviewer-caught trainer bugs** (fixed before any real run): an `--lr` override silently discarded on resume but logged as applied; a resumed finished run training one extra step at lr 0; orphan metric rows after a hard crash; 0 occurrences of `[CLS]` in the packed MLM stream (every window now starts with `[CLS]`, so the classifier's pooled position is pretrained).
+- **DataLoader migration with an RNG side effect.** The course requires `torch.utils.data.DataLoader` for batching and shuffling; `finetune.py` had its own numpy sampler. Moving every batch through `DataLoader` (row-index Dataset + seeded length-bucketed `batch_sampler` + padding collate, Step 2) was verified loss-identical at every step on the CPU fixture, except that iterating a DataLoader draws its worker base seed from the global torch RNG and shifted the dropout stream (step-1 loss 1.1093 vs 1.1125). Passing an explicit `generator` removed the side effect.
+- **Self-test artifacts are not results.** The agents' dev checkpoints (scratch init, a handful of steps on 2,000 rows) and their `experiments.jsonl` lines were deleted before the real queue was armed, so nothing in `reports/` or `checkpoints/` can be mistaken for a result; this notebook labels anything named `dev*` and shows "not available yet" until the queue writes the real artifacts.
+""")
+code("""
+shutil.rmtree(NOTEBOOK_TMP, ignore_errors=True)      # the save round-trip files and the staged symlinks (Step 5 / Step 7)
+elapsed = timer() - T_NOTEBOOK_START
+print(f"Notebook executed top-to-bottom in {elapsed:.0f} s on {device}; scratch dir {NOTEBOOK_TMP.name} removed: {not NOTEBOOK_TMP.exists()}.")
+if MISSING:
+    print(f"\\n{len(MISSING)} artifact(s) not available yet (they appear as the pipeline advances):")
+    for rel, by in dict(MISSING).items():
+        print(f"  - {rel}\\n      produced by: {by}")
+else:
+    print("Every artifact was present: this is the final run.")
+""")
+
+
+def build() -> nbformat.NotebookNode:
+    nb = new_notebook()
+    nb.metadata = {
+        "kernelspec": {"display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3"},
+        "language_info": {"name": "python"},
+    }
+    for i, (kind, text) in enumerate(CELLS):
+        cell = new_markdown_cell(text) if kind == "markdown" else new_code_cell(text)
+        cell["id"] = f"cell-{i:02d}"   # deterministic ids (nbformat would draw random ones), so rebuilds are byte-identical
+        nb.cells.append(cell)
+    return nb
+
+
+def main() -> int:
+    nb = build()
+    nbformat.validate(nb)
+    nbformat.write(nb, OUT)
+    n_md = sum(1 for k, _ in CELLS if k == "markdown")
+    print(f"wrote {OUT} ({len(CELLS)} cells: {n_md} markdown, {len(CELLS) - n_md} code)")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/presentation/scripts/launch_pretrain.sh b/presentation/scripts/launch_pretrain.sh
new file mode 100644
index 0000000000000000000000000000000000000000..9c8343837f2c5187d1459123ec061ef90c41ad83
--- /dev/null
+++ b/presentation/scripts/launch_pretrain.sh
@@ -0,0 +1,33 @@
+#!/usr/bin/env bash
+# Launch (or resume) the MLM pretraining run inside a detached tmux session.
+#
+#   scripts/launch_pretrain.sh                 # fresh run: EPOCHS=6 MICRO_BATCH=16 RUN_NAME=large-24L
+#   EPOCHS=3 scripts/launch_pretrain.sh        # shorter plan (fixes the LR schedule length)
+#   RESUME=1 scripts/launch_pretrain.sh        # continue the same run from its latest checkpoint
+#   tmux attach -t phish-pretrain              # watch; Ctrl-b d to detach; Ctrl-C inside = save + clean exit
+#
+# The schedule length is fixed at the first launch (epochs x steps/epoch); a resume continues it.
+# Measured on 2026-10-07: micro-batch 16 -> 13.65 GiB reserved, ~23.6k tokens/s, ~1.25 h per epoch.
+set -euo pipefail
+cd "$(dirname "$0")/.."
+RUN_NAME="${RUN_NAME:-large-24L}"
+EPOCHS="${EPOCHS:-6}"
+MICRO_BATCH="${MICRO_BATCH:-16}"
+SESSION="${SESSION:-phish-pretrain}"
+PY=".venv/bin/python"
+LOG="logs/pretrain_${RUN_NAME}_$(date +%Y%m%d-%H%M%S).out"
+
+if tmux has-session -t "$SESSION" 2>/dev/null; then
+  echo "tmux session '$SESSION' already exists; attach with: tmux attach -t $SESSION"; exit 1
+fi
+if [[ "${RESUME:-0}" == "1" ]]; then
+  CMD="$PY src/pretrain.py train --run-name $RUN_NAME --resume --micro-batch $MICRO_BATCH"
+else
+  CMD="$PY src/pretrain.py train --run-name $RUN_NAME --epochs $EPOCHS --micro-batch $MICRO_BATCH"
+fi
+echo "launching in tmux '$SESSION': $CMD"
+echo "stdout/stderr -> $LOG (the trainer also writes logs/pretrain-<timestamp>.log and checkpoints/pretrain/$RUN_NAME/metrics.csv)"
+tmux new-session -d -s "$SESSION" -c "$PWD" "$CMD 2>&1 | tee '$LOG'; echo; echo '[pretrain exited with status '\${PIPESTATUS[0]}'] press Enter to close'; read"
+sleep 2
+tmux capture-pane -p -t "$SESSION" | tail -5
+echo "attach: tmux attach -t $SESSION"
diff --git a/presentation/scripts/run_experiments.sh b/presentation/scripts/run_experiments.sh
new file mode 100644
index 0000000000000000000000000000000000000000..57b9e3a2b52d2c27d7c5eb6b6863451104788390
--- /dev/null
+++ b/presentation/scripts/run_experiments.sh
@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+# Fine-tuning experiment queue (the instructor's dials: loss function, epochs, layers, plus the
+# pretraining and synthetic-data ablations). Waits for the pretraining run to finish, then runs
+# each experiment sequentially on the GPU, evaluates its best/ on the TREC-06 binary holdout and
+# the three-way holdout, and exports the main run. Launch in tmux:
+#
+#   tmux new-session -d -s phish-finetune 'scripts/run_experiments.sh 2>&1 | tee logs/experiments_$(date +%Y%m%d-%H%M%S).out'
+#
+# Resumable: a run whose best/ already exists is skipped; re-run the script to continue after a crash.
+set -uo pipefail
+cd "$(dirname "$0")/.."
+PY=".venv/bin/python"
+PRETRAIN="checkpoints/pretrain/${PRETRAIN_RUN:-large-24L}/final"
+EVAL_EVERY="${EVAL_EVERY:-1250}"          # ~quarter epoch at batch 32 over 160k rows (val eval costs ~1 min)
+MAIN="${MAIN_RUN:-ft-ce}"
+
+log() { printf '%s %s\n' "$(date +%H:%M:%S)" "$*"; }
+
+if [[ ! -f "$PRETRAIN/model.safetensors" ]]; then
+  log "waiting for $PRETRAIN/model.safetensors (pretraining still running)"
+  while [[ ! -f "$PRETRAIN/model.safetensors" ]]; do sleep 300; done
+  log "pretraining final/ found; waiting 60 s for the GPU to free"; sleep 60
+fi
+$PY src/pretrain.py plot --run-name "${PRETRAIN_RUN:-large-24L}" || true
+
+# name | extra flags            (all: --init pretrained:$PRETRAIN unless --init scratch is given)
+EXPERIMENTS=(
+  "ft-ce        --loss cross_entropy --epochs 3"
+  "ft-weighted  --loss weighted_cross_entropy --epochs 2"
+  "ft-focal     --loss focal --epochs 2"
+  "ft-synthetic --loss cross_entropy --epochs 2 --synthetic on"
+  "ft-12L       --loss cross_entropy --epochs 2 --n-layers 12"
+  "ft-scratch   --loss cross_entropy --epochs 2 --init scratch"
+)
+STATUS=()
+for spec in "${EXPERIMENTS[@]}"; do
+  name="${spec%% *}"; flags="${spec#* }"
+  if [[ -f "checkpoints/finetune/$name/best/model.safetensors" ]]; then
+    log "$name: best/ exists, skipping training"
+  else
+    init="--init pretrained:$PRETRAIN"; [[ "$flags" == *"--init scratch"* ]] && init=""
+    log "$name: $PY src/finetune.py train --run-name $name $init $flags --eval-every $EVAL_EVERY"
+    $PY src/finetune.py train --run-name "$name" $init $flags --eval-every "$EVAL_EVERY"
+    rc=$?; log "$name: finetune exit $rc"; STATUS+=("$name:train=$rc")
+    [[ $rc -ne 0 ]] && continue
+  fi
+  $PY src/finetune.py plot --run-name "$name" --out "reports/finetune_curves_$name.png" || true
+  if [[ ! -f "reports/eval_done_$name" ]]; then
+    # one invocation = one experiments.jsonl line + one RUN_LOG entry covering both holdouts
+    $PY src/evaluate.py run --checkpoint "checkpoints/finetune/$name/best" --split test,test3 --run-name "$name" --out-dir "reports/eval_$name"
+    rc=$?; log "$name: evaluate exit $rc"; STATUS+=("$name:eval=$rc")
+    [[ $rc -eq 0 ]] && touch "reports/eval_done_$name"
+  fi
+done
+# main run: top-level reports + export + app files
+if [[ -f "checkpoints/finetune/$MAIN/best/model.safetensors" ]]; then
+  $PY src/finetune.py plot --run-name "$MAIN" || true
+  $PY src/evaluate.py run --checkpoint "checkpoints/finetune/$MAIN/best" --split test,test3 --run-name "$MAIN" \
+      --finetune-run-dir "checkpoints/finetune/$MAIN" --pretrain-run-dir "checkpoints/pretrain/${PRETRAIN_RUN:-large-24L}"
+  $PY src/export.py run --checkpoint "checkpoints/finetune/$MAIN/best" --out-dir "model_export/$MAIN" --app-dir app
+  STATUS+=("$MAIN:export=$?")
+fi
+$PY src/finetune.py list
+log "queue finished: ${STATUS[*]:-nothing ran}"
diff --git a/presentation/scripts/upload_to_hf.py b/presentation/scripts/upload_to_hf.py
new file mode 100644
index 0000000000000000000000000000000000000000..447f9b40c084f8ffdd306751571a0522d848a790
--- /dev/null
+++ b/presentation/scripts/upload_to_hf.py
@@ -0,0 +1,284 @@
+"""Publish the exported classifier and the Gradio demo to Hugging Face (private model repo + private Space).
+
+NATHAN RUNS THIS HIMSELF, AFTER EXPLICIT APPROVAL. No agent, hook or workflow may invoke it with
+``--yes``. The default is a dry run that prints the exact plan (files, sizes, repos, hardware,
+visibility, secret and variable names) and performs no network call at all; ``--check-auth`` adds a
+single read-only ``whoami`` to the dry run. Anything that writes to the Hub requires ``--yes``.
+
+What a live run does, in order (huggingface_hub 1.33.0 API, verified in docs/research):
+
+1. ``HfApi().whoami()`` -- the token comes from ``HF_TOKEN`` or the ``hf auth login`` cache and is never
+   printed; only the account name is logged.
+2. ``create_repo(model_repo, repo_type="model", private=True, exist_ok=True)``.
+3. ``upload_folder(model_dir -> model_repo, allow_patterns=[model.safetensors, config.json,
+   tokenizer.json, README.md])``: the bf16 safetensors export, its config, the tokenizer and the model card.
+4. ``create_repo(space_repo, repo_type="space", space_sdk="gradio", space_hardware=<--hardware>,
+   private=True, exist_ok=True)``.
+5. ``add_space_secret(space_repo, "HF_TOKEN", $HF_TOKEN_FOR_SPACE)``: a READ token the Space uses to download
+   the private weights at startup (``preload_from_hub`` cannot read private repos). The value must come from
+   the environment variable ``HF_TOKEN_FOR_SPACE``; the script never prompts and never prints it. Use
+   ``--skip-secret`` if the secret is already set.
+6. ``add_space_variable(space_repo, "PHISH_MODEL_REPO", model_repo)`` so app.py knows where to download from.
+7. ``upload_folder(app_dir -> space_repo)``, by default ignoring ``*.safetensors``, ``tokenizer.json``,
+   ``config.json``, ``__pycache__/*`` and ``*.pyc`` so the Space fetches weights from the model repo at
+   runtime; ``--bundle-weights`` uploads them into the Space instead (then no secret is needed).
+8. Print both URLs and ``get_space_runtime(space_repo).hardware`` to confirm the ZeroGPU flavour stuck.
+
+Example (dry run, the default):   .venv/bin/python scripts/upload_to_hf.py --model-dir model_export/<run>
+Example (live, after approval):   HF_TOKEN_FOR_SPACE=hf_... .venv/bin/python scripts/upload_to_hf.py \\
+                                      --model-dir model_export/<run> --yes
+"""
+
+from __future__ import annotations
+
+import argparse
+import logging
+import os
+import sys
+from dataclasses import dataclass, field
+from pathlib import Path
+from typing import Sequence
+
+sys.path.insert(0, str(Path(__file__).resolve().parent.parent / "src"))
+
+import common  # noqa: E402
+
+LOG = logging.getLogger("upload_to_hf")
+
+MODEL_ALLOW_PATTERNS: list[str] = ["model.safetensors", "config.json", "tokenizer.json", "README.md"]
+SPACE_IGNORE_PATTERNS: list[str] = ["*.safetensors", "tokenizer.json", "config.json", "__pycache__/*", "*.pyc"]
+SPACE_REQUIRED_FILES: tuple[str, ...] = ("app.py", "model.py", "requirements.txt", "README.md")
+SECRET_ENV = "HF_TOKEN_FOR_SPACE"
+
+
+@dataclass
+class Plan:
+    model_dir: Path
+    app_dir: Path
+    model_repo: str
+    space_repo: str
+    hardware: str
+    private: bool
+    bundle_weights: bool
+    skip_secret: bool
+    model_files: list[tuple[str, int]] = field(default_factory=list)
+    space_files: list[tuple[str, int]] = field(default_factory=list)
+    problems: list[str] = field(default_factory=list)
+
+    @property
+    def space_ignore(self) -> list[str]:
+        return ["__pycache__/*", "*.pyc"] if self.bundle_weights else list(SPACE_IGNORE_PATTERNS)
+
+
+def _rel_files(folder: Path) -> list[str]:
+    return sorted(str(p.relative_to(folder)) for p in folder.rglob("*") if p.is_file() and ".git" not in p.parts)
+
+
+def _select(folder: Path, allow: Sequence[str] | None, ignore: Sequence[str] | None) -> list[tuple[str, int]]:
+    """Apply the same include/exclude logic ``upload_folder`` uses, so the dry run lists exactly what would ship."""
+    from huggingface_hub.utils import filter_repo_objects
+
+    chosen = filter_repo_objects(_rel_files(folder), allow_patterns=list(allow) if allow else None,
+                                 ignore_patterns=list(ignore) if ignore else None)
+    return [(rel, (folder / rel).stat().st_size) for rel in chosen]
+
+
+def build_plan(args: argparse.Namespace) -> Plan:
+    plan = Plan(
+        model_dir=Path(args.model_dir).resolve(), app_dir=Path(args.app_dir).resolve(),
+        model_repo=args.model_repo, space_repo=args.space_repo, hardware=args.hardware,
+        private=args.private, bundle_weights=args.bundle_weights, skip_secret=args.skip_secret,
+    )
+    if not plan.model_dir.is_dir():
+        plan.problems.append(f"model dir missing: {plan.model_dir} (run export.py first)")
+    else:
+        plan.model_files = _select(plan.model_dir, MODEL_ALLOW_PATTERNS, None)
+        present = {rel for rel, _ in plan.model_files}
+        for f in MODEL_ALLOW_PATTERNS[:3]:
+            if f not in present:
+                plan.problems.append(f"model dir lacks {f}")
+    if not plan.app_dir.is_dir():
+        plan.problems.append(f"app dir missing: {plan.app_dir}")
+    else:
+        plan.space_files = _select(plan.app_dir, None, plan.space_ignore)
+        present = {rel for rel, _ in plan.space_files}
+        for f in SPACE_REQUIRED_FILES:
+            if f not in present:
+                plan.problems.append(f"app dir lacks {f}")
+        if plan.bundle_weights:
+            for f in ("model.safetensors", "config.json", "tokenizer.json"):
+                if f not in present:
+                    plan.problems.append(f"--bundle-weights but app dir lacks {f} (src/export.py run copies them into the app dir)")
+        src_model = common.project_root() / "src" / "model.py"
+        app_model = plan.app_dir / "model.py"
+        if src_model.is_file() and app_model.is_file() and src_model.read_bytes() != app_model.read_bytes():
+            plan.problems.append("app/model.py is not byte-identical to src/model.py")
+    if not plan.bundle_weights and not plan.skip_secret and not os.environ.get(SECRET_ENV):
+        plan.problems.append(
+            f"{SECRET_ENV} is not set. The Space needs a READ token stored as the HF_TOKEN secret to download "
+            f"the private weights at startup. Export it in the shell that runs this script (never commit it), "
+            f"or pass --skip-secret if the secret already exists on the Space, or --bundle-weights."
+        )
+    return plan
+
+
+def _fmt_size(n: int) -> str:
+    for unit in ("B", "KiB", "MiB", "GiB"):
+        if n < 1024 or unit == "GiB":
+            return f"{n:,.0f} {unit}" if unit == "B" else f"{n:.1f} {unit}"
+        n /= 1024
+    return f"{n:.1f} GiB"
+
+
+def print_plan(plan: Plan, dry_run: bool) -> None:
+    vis = "private" if plan.private else "PUBLIC"
+    print(f"\n{'DRY RUN - nothing will be sent' if dry_run else 'LIVE RUN'}")
+    print(f"account token source: {'HF_TOKEN env' if os.environ.get('HF_TOKEN') else 'hf auth login cache'} (value never printed)")
+    print(f"\n[1] model repo  {plan.model_repo}  ({vis}, repo_type=model, exist_ok)")
+    print(f"    upload_folder({plan.model_dir}) allow_patterns={MODEL_ALLOW_PATTERNS}")
+    for rel, size in plan.model_files:
+        print(f"      {rel:<24} {_fmt_size(size):>12}")
+    print(f"    total {_fmt_size(sum(s for _, s in plan.model_files))} in {len(plan.model_files)} files")
+    print(f"\n[2] space repo  {plan.space_repo}  ({vis}, sdk=gradio, hardware={plan.hardware}, exist_ok)")
+    if plan.bundle_weights:
+        print("    secret HF_TOKEN: not needed (weights bundled)")
+    elif plan.skip_secret:
+        print("    secret HF_TOKEN: skipped (--skip-secret)")
+    else:
+        print(f"    secret HF_TOKEN  <- ${SECRET_ENV} ({'set' if os.environ.get(SECRET_ENV) else 'NOT SET'})")
+    print(f"    variable PHISH_MODEL_REPO = {plan.model_repo}")
+    print(f"    upload_folder({plan.app_dir}) ignore_patterns={plan.space_ignore}")
+    for rel, size in plan.space_files:
+        print(f"      {rel:<24} {_fmt_size(size):>12}")
+    print(f"    total {_fmt_size(sum(s for _, s in plan.space_files))} in {len(plan.space_files)} files")
+    print(f"\n    urls: https://huggingface.co/{plan.model_repo}  https://huggingface.co/spaces/{plan.space_repo}")
+    if plan.problems:
+        print("\nPROBLEMS (a live run refuses to start while any remains):")
+        for p in plan.problems:
+            print(f"  - {p}")
+    print()
+
+
+def check_auth() -> str:
+    """Read-only identity check; logs the account name only."""
+    from huggingface_hub import HfApi
+
+    me = HfApi().whoami()
+    name = me.get("name", "?")
+    LOG.info("authenticated as %s (type=%s)", name, me.get("type"))
+    return name
+
+
+def run_live(plan: Plan, commit_message: str) -> int:
+    from huggingface_hub import HfApi
+
+    api = HfApi()
+    name = check_auth()
+    owner = plan.model_repo.split("/")[0]
+    if name != owner:
+        LOG.warning("token belongs to %s but repos are under %s; the push will fail unless %s has write access", name, owner, name)
+
+    LOG.info("create_repo %s (model, private=%s)", plan.model_repo, plan.private)
+    url = api.create_repo(plan.model_repo, repo_type="model", private=plan.private, exist_ok=True)
+    LOG.info("model repo: %s", url)
+    LOG.info("upload_folder %s -> %s", plan.model_dir, plan.model_repo)
+    info = api.upload_folder(
+        folder_path=str(plan.model_dir), repo_id=plan.model_repo, repo_type="model",
+        commit_message=commit_message, allow_patterns=MODEL_ALLOW_PATTERNS,
+    )
+    LOG.info("model commit: %s", getattr(info, "commit_url", info))
+
+    LOG.info("create_repo %s (space, sdk=gradio, hardware=%s, private=%s)", plan.space_repo, plan.hardware, plan.private)
+    url = api.create_repo(
+        plan.space_repo, repo_type="space", space_sdk="gradio", space_hardware=plan.hardware,
+        private=plan.private, exist_ok=True,
+    )
+    LOG.info("space repo: %s", url)
+    if not plan.bundle_weights and not plan.skip_secret:
+        api.add_space_secret(plan.space_repo, "HF_TOKEN", os.environ[SECRET_ENV],
+                             description="read token for the private model repo")
+        LOG.info("space secret HF_TOKEN set")
+    api.add_space_variable(plan.space_repo, "PHISH_MODEL_REPO", plan.model_repo,
+                           description="model repo app.py downloads weights from")
+    LOG.info("space variable PHISH_MODEL_REPO=%s set", plan.model_repo)
+    LOG.info("upload_folder %s -> %s", plan.app_dir, plan.space_repo)
+    info = api.upload_folder(
+        folder_path=str(plan.app_dir), repo_id=plan.space_repo, repo_type="space",
+        commit_message=commit_message, ignore_patterns=plan.space_ignore,
+    )
+    LOG.info("space commit: %s", getattr(info, "commit_url", info))
+
+    runtime = api.get_space_runtime(plan.space_repo)
+    hardware = getattr(runtime, "hardware", None)
+    requested = getattr(runtime, "requested_hardware", None)
+    print(f"\nmodel: https://huggingface.co/{plan.model_repo}")
+    print(f"space: https://huggingface.co/spaces/{plan.space_repo}")
+    print(f"space runtime: stage={getattr(runtime, 'stage', '?')} hardware={hardware} requested={requested}")
+    if hardware != plan.hardware and requested != plan.hardware:
+        LOG.warning("hardware is %s, not %s: set it in Space settings or api.request_space_hardware()", hardware, plan.hardware)
+    return 0
+
+
+def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0], formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--config", default="config.yaml")
+    ap.add_argument("--model-dir", default=None, help="export dir (model_export/<run>); default: newest under model_export/")
+    ap.add_argument("--app-dir", default=None, help="Space source dir (default: config paths.app_dir)")
+    ap.add_argument("--model-repo", default=None, help="default: config hf.model_repo")
+    ap.add_argument("--space-repo", default=None, help="default: config hf.space_repo")
+    ap.add_argument("--hardware", default=None, help="Space hardware flavour (default: config hf.space_hardware)")
+    ap.add_argument("--private", action=argparse.BooleanOptionalAction, default=True)
+    ap.add_argument("--dry-run", action=argparse.BooleanOptionalAction, default=None,
+                    help="print the plan only (DEFAULT). --no-dry-run alone is refused; add --yes")
+    ap.add_argument("--yes", action="store_true", help="actually write to the Hub (Nathan only, after approval)")
+    ap.add_argument("--check-auth", action="store_true", help="dry run: also call whoami() (read-only)")
+    ap.add_argument("--bundle-weights", action="store_true", help="upload weights into the Space instead of the model repo download")
+    ap.add_argument("--skip-secret", action="store_true", help="do not set the HF_TOKEN space secret (already set)")
+    ap.add_argument("--commit-message", default="Upload BITS phishing detector")
+    return ap.parse_args(argv)
+
+
+def _newest_export(root: Path) -> Path | None:
+    if not root.is_dir():
+        return None
+    runs = [p for p in root.iterdir() if p.is_dir()]
+    return max(runs, key=lambda p: p.stat().st_mtime) if runs else None
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    args = parse_args(argv)
+    cfg = common.load_config(args.config)
+    global LOG
+    LOG = common.setup_logging("upload_to_hf", cfg["paths"]["logs"])
+    hf = cfg.get("hf", {})
+    root = common.project_root()
+    args.model_repo = args.model_repo or hf.get("model_repo", "BuildITSystems/phishing-detector-encoder")
+    args.space_repo = args.space_repo or hf.get("space_repo", "BuildITSystems/phishing-detector-demo")
+    args.hardware = args.hardware or hf.get("space_hardware", "zero-a10g")
+    args.app_dir = args.app_dir or cfg["paths"].get("app_dir", str(root / "app"))
+    if args.model_dir is None:
+        newest = _newest_export(root / "model_export")
+        args.model_dir = str(newest) if newest else str(root / "model_export" / "<run_name>")
+
+    dry_run = args.dry_run if args.dry_run is not None else not args.yes
+    if not dry_run and not args.yes:
+        LOG.error("--no-dry-run requires --yes; refusing")
+        return 2
+    if dry_run and args.yes:
+        LOG.info("--dry-run given together with --yes: dry run wins")
+
+    plan = build_plan(args)
+    print_plan(plan, dry_run)
+    if dry_run:
+        if args.check_auth:
+            check_auth()
+        LOG.info("dry run complete; no Hub write performed")
+        return 0
+    if plan.problems:
+        LOG.error("refusing live run: %d problem(s) listed above", len(plan.problems))
+        return 2
+    return run_live(plan, args.commit_message)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
diff --git a/presentation/scripts/watch_pretrain.sh b/presentation/scripts/watch_pretrain.sh
new file mode 100644
index 0000000000000000000000000000000000000000..bb7d5e78b2a8a04032cb3196441b9c754a84dab1
--- /dev/null
+++ b/presentation/scripts/watch_pretrain.sh
@@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+# Pretraining watchdog. Every 2 minutes: if final/ exists -> done. If the trainer process is gone and
+# its last recorded status is a RECOVERABLE stop (oom, vram_guard, or 'running' = hard crash), relaunch
+# with RESUME=1 (at most MAX_RETRIES times, 2 min apart so a desktop VRAM spike can pass). Never resumes
+# after a deliberate stop (signal:SIGINT/SIGTERM), a normal end, or a divergence: those need a human.
+#
+#   tmux new-session -d -s phish-watchdog 'scripts/watch_pretrain.sh 2>&1 | tee -a logs/watchdog.out'
+set -uo pipefail
+cd "$(dirname "$0")/.."
+RUN="${RUN_NAME:-large-24L}"
+MAX="${MAX_RETRIES:-3}"
+MB="${MICRO_BATCH:-24}"
+n=0
+log() { printf '%s %s\n' "$(date '+%F %T')" "$*"; }
+log "watchdog armed for $RUN (max $MAX auto-resumes, micro-batch $MB)"
+while true; do
+  if [[ -f "checkpoints/pretrain/$RUN/final/model.safetensors" ]]; then
+    log "final/ exists: pretraining complete; watchdog exiting"; exit 0
+  fi
+  # anchored to the interpreter so the tmux server and the `bash -c ... | tee` wrapper (whose argv
+  # also contain the command and outlive the trainer) do not count as a running trainer
+  if pgrep -f "^[^ ]*python[0-9.]* src/pretrain\.py train --run-name $RUN( |$)" >/dev/null; then
+    sleep 120; continue
+  fi
+  status=$(python3 -c "import json;print(json.load(open('checkpoints/pretrain/$RUN/run_summary.json')).get('status','?'))" 2>/dev/null || echo "?")
+  log "trainer not running; last recorded status '$status'"
+  case "$status" in
+    complete|max_steps|max_hours) log "run ended normally ($status) without final/; nothing to do"; exit 0 ;;
+    signal:*) log "stopped deliberately ($status); NOT resuming"; exit 0 ;;
+    diverged) log "run diverged; NOT resuming, needs a human"; exit 1 ;;
+    oom|vram_guard|running) : ;;
+    *) log "unknown status '$status'; NOT resuming"; exit 1 ;;
+  esac
+  if (( n >= MAX )); then log "retry limit $MAX reached; giving up"; exit 1; fi
+  n=$((n + 1))
+  log "auto-resume $n/$MAX in 120 s (GPU in use now: $(nvidia-smi --query-gpu=memory.used --format=csv,noheader))"
+  sleep 120
+  tmux kill-session -t phish-pretrain 2>/dev/null || true
+  RESUME=1 MICRO_BATCH="$MB" scripts/launch_pretrain.sh || log "launcher failed"
+  sleep 180
+done
diff --git a/presentation/src/common.py b/presentation/src/common.py
new file mode 100644
index 0000000000000000000000000000000000000000..4508b073bbcf39b1eea9cd4643659064f5318f4c
--- /dev/null
+++ b/presentation/src/common.py
@@ -0,0 +1,220 @@
+"""Shared helpers for the BITS Phishing Detector pipeline.
+
+Owned by the data-prep stage; every other src/*.py imports it as ``import common``.
+Only stdlib + yaml + numpy + torch are touched here so the module is cheap to import
+from CLI entry points, notebooks and tests alike.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import logging
+import os
+import platform
+import random
+import re
+import sys
+import timeit
+import unicodedata
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+import yaml
+
+_PATH_KEYS = frozenset(
+    {"data_raw", "data_processed", "tokenizer", "checkpoints", "reports", "logs", "app_dir"}
+)
+_NON_ALNUM = re.compile(r"[^a-z0-9]+")
+
+
+def project_root() -> Path:
+    """Parent of ``src/``; every relative path in config.yaml is anchored here."""
+    return Path(__file__).resolve().parent.parent
+
+
+def load_config(path: str | Path = "config.yaml") -> dict[str, Any]:
+    """yaml.safe_load of the project config with ``paths.*`` resolved against the project root.
+
+    Resolving at load time means every script can be launched from any cwd and still write
+    to the same data/, checkpoints/ and logs/ directories.
+    """
+    cfg_path = Path(path)
+    if not cfg_path.is_absolute():
+        cfg_path = project_root() / cfg_path
+    with cfg_path.open("r", encoding="utf-8") as fh:
+        cfg = yaml.safe_load(fh)
+    paths = cfg.get("paths") or {}
+    for key, value in list(paths.items()):
+        if key in _PATH_KEYS and isinstance(value, str) and not os.path.isabs(value):
+            paths[key] = str(project_root() / value)
+    cfg["paths"] = paths
+    cfg["_config_path"] = str(cfg_path)
+    return cfg
+
+
+def set_seed(seed: int) -> None:
+    """Seed random, numpy and torch (CPU + every CUDA device).
+
+    Deliberately does NOT call ``torch.use_deterministic_algorithms``: that would disable the
+    fast SDPA / cuBLAS kernels the 337M encoder depends on, and bitwise reproducibility is not a
+    project requirement (seeded splits + logged configs are).
+    """
+    random.seed(seed)
+    np.random.seed(seed)
+    try:
+        import torch
+
+        torch.manual_seed(seed)
+        if torch.cuda.is_available():
+            torch.cuda.manual_seed_all(seed)
+    except ImportError:  # pragma: no cover - torch is pinned, but keep common.py importable without it
+        pass
+
+
+def setup_logging(name: str, log_dir: str | Path = "logs") -> logging.Logger:
+    """Console + file logger (``logs/<name>-<YYYYmmdd-HHMMSS>.log``) at INFO.
+
+    Each script gets its own timestamped file so that re-runs never clobber the evidence of an
+    earlier run; RUN_LOG.md quotes from these files.
+    """
+    log_path = Path(log_dir)
+    if not log_path.is_absolute():
+        log_path = project_root() / log_path
+    log_path.mkdir(parents=True, exist_ok=True)
+    stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
+    logfile = log_path / f"{name}-{stamp}.log"
+
+    logger = logging.getLogger(name)
+    logger.setLevel(logging.INFO)
+    logger.propagate = False
+    for handler in list(logger.handlers):
+        logger.removeHandler(handler)
+    fmt = logging.Formatter("%(asctime)s %(levelname)s %(name)s: %(message)s", "%H:%M:%S")
+    stream = logging.StreamHandler(sys.stdout)
+    stream.setFormatter(fmt)
+    file_handler = logging.FileHandler(logfile, encoding="utf-8")
+    file_handler.setFormatter(fmt)
+    logger.addHandler(stream)
+    logger.addHandler(file_handler)
+    logger.info("log file: %s", logfile)
+    return logger
+
+
+def print_device_info() -> dict[str, Any]:
+    """Print and return the runtime facts that matter for VRAM planning; CPU-safe."""
+    info: dict[str, Any] = {
+        "python": platform.python_version(),
+        "torch": None,
+        "cuda_build": None,
+        "cudnn": None,
+        "device": "cpu",
+        "sm": None,
+        "vram_gib": 0.0,
+        "bf16": False,
+        "free_gib": 0.0,
+    }
+    try:
+        import torch
+
+        info["torch"] = torch.__version__
+        info["cuda_build"] = torch.version.cuda
+        info["cudnn"] = torch.backends.cudnn.version()
+        if torch.cuda.is_available():
+            idx = torch.cuda.current_device()
+            props = torch.cuda.get_device_properties(idx)
+            free, total = torch.cuda.mem_get_info(idx)
+            info.update(
+                device=props.name,
+                sm=f"{props.major}.{props.minor}",
+                vram_gib=round(total / 2**30, 2),
+                bf16=bool(torch.cuda.is_bf16_supported()),
+                free_gib=round(free / 2**30, 2),
+            )
+    except ImportError:
+        pass
+    for key, value in info.items():
+        print(f"{key:>10}: {value}")
+    return info
+
+
+def vram_stats() -> dict[str, float]:
+    """Current/peak allocated and reserved VRAM in GB (zeros on CPU)."""
+    zeros = {"allocated_gb": 0.0, "reserved_gb": 0.0, "max_allocated_gb": 0.0, "max_reserved_gb": 0.0}
+    try:
+        import torch
+
+        if not torch.cuda.is_available():
+            return zeros
+        gb = float(2**30)
+        return {
+            "allocated_gb": torch.cuda.memory_allocated() / gb,
+            "reserved_gb": torch.cuda.memory_reserved() / gb,
+            "max_allocated_gb": torch.cuda.max_memory_allocated() / gb,
+            "max_reserved_gb": torch.cuda.max_memory_reserved() / gb,
+        }
+    except ImportError:
+        return zeros
+
+
+def timer() -> float:
+    return timeit.default_timer()
+
+
+def normalize_text(s: str) -> str:
+    """Dedup normalisation: NFKC -> lowercase -> keep only [a-z0-9].
+
+    Removing every non-alphanumeric character neutralises the Enron-Spam " , " tokenisation,
+    whitespace/line-ending differences and HTML entity noise, so two renderings of the same email
+    collide on the same key. The NFKC step is an extension of the verify_data.md section 7 wording
+    ("lowercase, keep [a-z0-9]"): it folds full-width and ligature variants onto the same key and
+    only ever merges more rows, so counts differ marginally from that document (e.g. 16,260 vs
+    16,259 TREC-06 rows under a body-only filter). Every sha1 key in data/processed derives from
+    THIS function; downstream code (app/, notebooks) must import or copy it verbatim, never
+    re-derive it from the one-line description.
+    """
+    if not s:
+        return ""
+    return _NON_ALNUM.sub("", unicodedata.normalize("NFKC", s).lower())
+
+
+def sha1_hex(s: str) -> str:
+    return hashlib.sha1(s.encode("utf-8", "surrogatepass")).hexdigest()
+
+
+def write_json(obj: Any, path: str | Path) -> None:
+    p = Path(path)
+    p.parent.mkdir(parents=True, exist_ok=True)
+    with p.open("w", encoding="utf-8") as fh:
+        json.dump(obj, fh, indent=2, sort_keys=False, default=_json_default)
+        fh.write("\n")
+
+
+def read_json(path: str | Path) -> Any:
+    with Path(path).open("r", encoding="utf-8") as fh:
+        return json.load(fh)
+
+
+def append_run_log(markdown: str, path: str | Path = "RUN_LOG.md") -> None:
+    """Append a blank line + the markdown block to RUN_LOG.md (the project's running record)."""
+    p = Path(path)
+    if not p.is_absolute():
+        p = project_root() / p
+    with p.open("a", encoding="utf-8") as fh:
+        fh.write("\n" + markdown.rstrip("\n") + "\n")
+
+
+def _json_default(obj: Any) -> Any:
+    if isinstance(obj, (np.integer,)):
+        return int(obj)
+    if isinstance(obj, (np.floating,)):
+        return float(obj)
+    if isinstance(obj, np.ndarray):
+        return obj.tolist()
+    if isinstance(obj, Path):
+        return str(obj)
+    if isinstance(obj, datetime):
+        return obj.isoformat()
+    raise TypeError(f"not JSON serialisable: {type(obj).__name__}")
diff --git a/presentation/src/data_prep.py b/presentation/src/data_prep.py
new file mode 100644
index 0000000000000000000000000000000000000000..1c5c14e4994d5026c8bfaf24242a89e8570c9bc6
--- /dev/null
+++ b/presentation/src/data_prep.py
@@ -0,0 +1,2121 @@
+#!/usr/bin/env python
+"""Data preparation for the BITS Phishing Detector.
+
+Download -> normalise -> dedupe -> decontaminate -> split -> leakage-check -> manifests.
+Procedure and expected counts: docs/research/verify_data.md sections 7 and 8.
+
+Subcommands
+  inspect   features + 2 rows of all four datasets (streaming; nothing is fully downloaded)
+  build     produce data/processed/{pretrain,finetune_train,finetune_val,test_holdout}.parquet,
+            splits_manifest.json and data_report.md
+  check     re-run the leakage assertions on the written files and print the manifest summary
+  build3    THREE-WAY labels (RUN_LOG entry 6): add label3/label_set to the binary splits, ingest the Champa
+            phishing corpora (Nazario, Nigerian_Fraud) [+ bigshp if data.threeway.bigshp.enabled], carve the
+            clean-provenance test_holdout3.parquet, rewrite finetune_train/val + test_holdout (2 columns added)
+  check3    recompute every binary + three-way leakage assertion from the written files
+
+Leakage contract (hard failures, asserted identically by `build` and `check`): the held-out
+TREC-06 sub-source never shares an exact key (k_full / k_pre / k_sb / k_sb_full) or a >=3-shingle
+near-duplicate link with the fine-tune train/val splits or the MLM pretraining corpus; train and val
+never share a k_full; val never shares a key with the pretraining corpus; the holdout source never
+appears in train/val. Info-only counters: train/val near-dup links, train/pretrain key overlap,
+val/pretrain near-dup links.
+
+Three-way contract (build3 / check3): label3 in {0 legitimate, 1 phishing, 2 spam, -1 unwanted-but-unsplit},
+label_set in {legit, phish, spam, unwanted}; binary `label` unchanged. Binary split membership is unchanged
+except for the rows moved from finetune_val into test_holdout3. test_holdout3 is asserted to share zero keys and
+zero >=3-shingle links with train, val and the pretraining corpus and zero keys with the TREC-06 holdout.
+
+The assertions are an independent recomputation, not the removal bookkeeping: after every removal
+step the shingle index is rebuilt from the surviving rows (and the pretraining rows are re-derived
+from their stored `text` exactly as `check` does), so a bug in the removal logic surfaces as a
+failed assertion rather than a silent pass.
+"""
+
+from __future__ import annotations
+
+import argparse
+import hashlib
+import json
+import logging
+import os
+import re
+import resource
+import shutil
+import sys
+import unicodedata
+from collections import Counter, defaultdict
+from dataclasses import dataclass, field
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Iterable
+
+import numpy as np
+import pandas as pd
+import pyarrow as pa
+import pyarrow.parquet as pq
+
+sys.path.insert(0, str(Path(__file__).parent))
+import common  # noqa: E402
+
+LOG = logging.getLogger("data_prep")
+
+# ----------------------------------------------------------------------------- dials
+@dataclass
+class Dials:
+    """Dedup / pretraining thresholds. Read from config.yaml when present, else these defaults.
+
+    Optional config keys (all under `data`): `dedup.key_prefix_chars`, `dedup.key_prefix_min_len`,
+    `dedup.near_dup_shingle_cap`, `pretrain_min_alnum_chars`, `quote_strip_min_alnum`. The body-key
+    fallback threshold is always `data.min_alnum_chars` so the two cannot diverge.
+    """
+
+    key_prefix_chars: int = 300        # k_pre / k_sb hash the first N normalised chars
+    key_prefix_min_len: int = 80       # shorter bases use k_full as k_pre (a prefix would be the whole text)
+    body_key_min_alnum: int = 20       # below this the body alone is degenerate; key on subject+body instead
+    pretrain_min_alnum: int = 50       # verify_data.md section 7 step 2: corbt bodies with <50 alnum are dropped
+    quote_strip_min_alnum: int = 200   # a quote/forward cut is applied only if this much text survives above it
+    shingle_cap: int = 2000            # shingles shared by more rows than this are boilerplate and ignored
+
+    @classmethod
+    def from_config(cls, cfg: dict) -> "Dials":
+        d = cfg["data"]
+        dd = d.get("dedup", {})
+        return cls(
+            key_prefix_chars=int(dd.get("key_prefix_chars", cls.key_prefix_chars)),
+            key_prefix_min_len=int(dd.get("key_prefix_min_len", cls.key_prefix_min_len)),
+            body_key_min_alnum=int(d["min_alnum_chars"]),
+            pretrain_min_alnum=int(d.get("pretrain_min_alnum_chars", cls.pretrain_min_alnum)),
+            quote_strip_min_alnum=int(d.get("quote_strip_min_alnum", cls.quote_strip_min_alnum)),
+            shingle_cap=int(dd.get("near_dup_shingle_cap", cls.shingle_cap)),
+        )
+
+
+DIALS = Dials()   # replaced by Dials.from_config(cfg) in main(); module-level so row_keys stays a plain function
+
+LABELLED_COLUMNS = ["text", "subject", "label", "dataset_name"]
+PRETRAIN_COLUMNS = ["subject", "body"]
+FINETUNE_SCHEMA = ["doc_id", "text", "subject", "body", "label", "source", "cluster_id"]
+PRETRAIN_SCHEMA = ["doc_id", "text", "n_chars"]
+
+_CRLF = re.compile(r"\r\n?")
+_BLANK_RUN = re.compile(r"\n(?:[ \t]*\n){2,}")   # 3+ newlines (2+ blank lines) -> one blank line
+_QUOTE_MARKERS = re.compile(
+    r"(?P<original>-----\s*Original Message\s*-----)"
+    r"|(?P<forwarded>-{3,}\s*Forwarded by\b)"
+    r"|(?P<header_block>^From:[^\n]*\n(?:[^\n]*\n){0,2}(?:Sent|To|Subject):)",
+    re.MULTILINE,
+)
+
+
+class SpamPolicyError(ValueError):
+    """Raised for spam policies that the available corpora cannot implement."""
+
+
+# ----------------------------------------------------------------------------- text utils
+def clean_text(s: Any) -> str:
+    """NFKC, normalised line endings, collapsed blank-line runs, stripped.
+
+    Keeps the email readable for a byte-level BPE (no aggressive stripping of punctuation or
+    HTML) while removing the formatting noise that would otherwise make identical emails look
+    different to the tokenizer and inflate the near-duplicate count.
+    """
+    if not isinstance(s, str) or not s:
+        return ""
+    s = unicodedata.normalize("NFKC", s).replace("\x00", "")
+    s = _CRLF.sub("\n", s)
+    s = _BLANK_RUN.sub("\n\n", s)
+    return s.strip()
+
+
+def model_input(subject: str, body: str) -> str:
+    """subject + blank line + body. Never sender/receiver/date (TREC headers are synthetic)."""
+    return "\n\n".join(part for part in (subject, body) if part)
+
+
+def strip_quotes(body: str) -> tuple[str, str | None, bool]:
+    """Cut a corbt body at its first quote/forward marker when >=200 alphanumerics survive.
+
+    Enron threads quote the whole history, so without this ~43% of pretraining characters would
+    be verbatim repeats of other rows. Returns (body, marker_name, cut_applied).
+    """
+    m = _QUOTE_MARKERS.search(body)
+    if m is None:
+        return body, None, False
+    head = body[: m.start()].rstrip()
+    if len(common.normalize_text(head)) >= DIALS.quote_strip_min_alnum:
+        return head, m.lastgroup, True
+    return body, m.lastgroup, False
+
+
+def row_keys(subject: str, body: str) -> tuple[str, str, str, str, str, int]:
+    """Dedup keys per verify_data.md section 7.
+
+    Returns (k_full, k_pre, k_sb, k_sb_full, base, n_alnum):
+      k_full    sha1 of the normalised body (the content-level identity of the email)
+      k_pre     sha1 of its first 300 chars (catches appended footers / truncated copies), or
+                k_full when the body is shorter than 80 chars
+      k_sb      sha1 of the first 300 chars of normalised subject+body (catches corpora that
+                embed the subject in the body, e.g. Enron-Spam, and corbt subject+body)
+      k_sb_full sha1 of the whole normalised subject+body (corbt dedup key)
+      base      the normalised string used for k_full/k_pre and shingles
+      n_alnum   alphanumerics in subject+body (the min_alnum_chars filter)
+    When the body holds fewer than `min_alnum_chars` alphanumerics the subject+body string is used
+    as `base` instead, otherwise every empty-body email would collide on sha1("").
+    """
+    norm_body = common.normalize_text(body)
+    norm_sb = common.normalize_text(subject) + norm_body
+    base = norm_body if len(norm_body) >= DIALS.body_key_min_alnum else norm_sb
+    k_full = common.sha1_hex(base)
+    k_pre = common.sha1_hex(base[: DIALS.key_prefix_chars]) if len(base) >= DIALS.key_prefix_min_len else k_full
+    k_sb = common.sha1_hex(norm_sb[: DIALS.key_prefix_chars])
+    k_sb_full = common.sha1_hex(norm_sb)
+    return k_full, k_pre, k_sb, k_sb_full, base, len(norm_sb)
+
+
+def split_text(text: str) -> tuple[str, str]:
+    """Recover (subject, body) from a stored `text` = model_input(subject, body).
+
+    Exact whenever the subject was non-empty (subjects never contain a blank line). For an
+    empty-subject email with a multi-paragraph body the first paragraph is mistaken for the
+    subject; `text_identity` is therefore only a lower bound on the build-time identity, and
+    build_pretrain compensates by matching on BOTH identities for empty-subject rows so the
+    post-hoc assertion (shared with `check`) can never see a link the removal pass did not.
+    """
+    head, sep, rest = text.partition("\n\n")
+    return (head, rest) if sep else ("", head)
+
+
+def text_identity(text: str) -> tuple[set[str], str]:
+    """(keys, shingle base) of a stored pretraining `text`, recomputed the same way by build and check.
+
+    normalize(subject + "\\n\\n" + body) == normalize(subject) + normalize(body), so k_sb / k_sb_full
+    are always exact; k_full / k_pre / base are exact unless the subject was empty (see split_text).
+    """
+    k_full, k_pre, k_sb, k_sb_full, base, _ = row_keys(*split_text(text))
+    return {k_full, k_pre, k_sb, k_sb_full}, base
+
+
+def add_key_columns(df: pd.DataFrame) -> list[str]:
+    """Attach k_full/k_pre/k_sb/k_sb_full/n_alnum columns; return the shingle bases (window-trimmed)."""
+    cols: dict[str, list] = defaultdict(list)
+    bases: list[str] = []
+    for subj, body in zip(df["subject"].tolist(), df["body"].tolist()):
+        k_full, k_pre, k_sb, k_sb_full, base, n_alnum = row_keys(subj, body)
+        cols["k_full"].append(k_full)
+        cols["k_pre"].append(k_pre)
+        cols["k_sb"].append(k_sb)
+        cols["k_sb_full"].append(k_sb_full)
+        cols["n_alnum"].append(n_alnum)
+        bases.append(base)
+    for name, values in cols.items():
+        df[name] = values
+    return bases
+
+
+def key_union(df: pd.DataFrame) -> set[str]:
+    out: set[str] = set()
+    for col in ("k_full", "k_pre", "k_sb", "k_sb_full"):
+        out.update(df[col].tolist())
+    return out
+
+
+def rows_hitting(df: pd.DataFrame, keys: set[str]) -> np.ndarray:
+    """Boolean mask: row has any of its four keys inside `keys`."""
+    mask = np.zeros(len(df), dtype=bool)
+    for col in ("k_full", "k_pre", "k_sb", "k_sb_full"):
+        mask |= np.fromiter((k in keys for k in df[col].tolist()), dtype=bool, count=len(df))
+    return mask
+
+
+# ----------------------------------------------------------------------------- near-dup index
+@dataclass
+class NearDupConfig:
+    shingle_chars: int = 50
+    stride: int = 25
+    window_chars: int = 1500
+    min_shared: int = 3
+    cap: int = 2000
+
+    @classmethod
+    def from_config(cls, cfg: dict, cap: int) -> "NearDupConfig":
+        d = cfg["data"]["dedup"]
+        return cls(
+            shingle_chars=int(d["near_dup_shingle_chars"]),
+            stride=int(d["near_dup_stride"]),
+            window_chars=int(d["near_dup_window_chars"]),
+            min_shared=int(d["near_dup_min_shared"]),
+            cap=cap,
+        )
+
+
+def shingle_hashes(base: str, nd: NearDupConfig) -> np.ndarray:
+    """Distinct 64-bit hashes of the fixed-stride shingles of base[:window]."""
+    s = base[: nd.window_chars]
+    n = len(s)
+    if n < nd.shingle_chars:
+        return np.empty(0, dtype=np.int64)
+    digests = b"".join(
+        hashlib.blake2b(s[i : i + nd.shingle_chars].encode("ascii"), digest_size=8).digest()
+        for i in range(0, n - nd.shingle_chars + 1, nd.stride)
+    )
+    return np.unique(np.frombuffer(digests, dtype=np.int64))
+
+
+class ShingleIndex:
+    """Inverted shingle index over a list of normalised texts.
+
+    Shingles shared by more than `cap` rows are boilerplate (list footers, pharma templates) and
+    are ignored: linking everything that carries the same footer would merge half a corpus into
+    one cluster and remove legitimate training rows. This is a known, logged limitation.
+    """
+
+    def __init__(self, bases: list[str], nd: NearDupConfig):
+        self.nd = nd
+        self.n = len(bases)
+        per_row = [shingle_hashes(b, nd) for b in bases]
+        lens = np.fromiter((len(h) for h in per_row), dtype=np.int64, count=self.n)
+        self.offsets = np.concatenate(([0], np.cumsum(lens)))
+        self.hashes = np.concatenate(per_row) if self.n else np.empty(0, dtype=np.int64)
+        rid = np.repeat(np.arange(self.n, dtype=np.int64), lens)
+        order = np.argsort(self.hashes, kind="stable")
+        self.sorted_rid = rid[order]
+        self.uniq, self.start, self.counts = np.unique(
+            self.hashes[order], return_index=True, return_counts=True
+        )
+        capped = self.counts > nd.cap
+        self.n_capped_shingles = int(capped.sum())
+        self.n_rows_touching_capped = int(
+            np.unique(self.sorted_rid[np.repeat(capped, self.counts)]).size
+        )
+        LOG.info(
+            "shingle index: %d rows, %d shingle occurrences, %d distinct, %d shingles over cap=%d "
+            "(ignored; touching %d rows)",
+            self.n, self.hashes.size, self.uniq.size, self.n_capped_shingles, nd.cap,
+            self.n_rows_touching_capped,
+        )
+
+    def links(self) -> tuple[np.ndarray, np.ndarray]:
+        """All pairs (a < b) sharing >= min_shared distinct shingles."""
+        out_a: list[np.ndarray] = []
+        out_b: list[np.ndarray] = []
+        t0 = common.timer()
+        for r in range(self.n):
+            h = self.hashes[self.offsets[r] : self.offsets[r + 1]]
+            if h.size == 0:
+                continue
+            g = np.searchsorted(self.uniq, h)
+            cnt = self.counts[g]
+            ok = (cnt >= 2) & (cnt <= self.nd.cap)
+            if not ok.any():
+                continue
+            starts = self.start[g[ok]]
+            lens = cnt[ok]
+            idx = np.repeat(starts - np.concatenate(([0], np.cumsum(lens)[:-1])), lens) + np.arange(lens.sum())
+            cand = self.sorted_rid[idx]
+            cand = cand[cand > r]
+            if cand.size < self.nd.min_shared:
+                continue
+            u, c = np.unique(cand, return_counts=True)
+            hit = u[c >= self.nd.min_shared]
+            if hit.size:
+                out_a.append(np.full(hit.size, r, dtype=np.int64))
+                out_b.append(hit)
+            if r and r % 50_000 == 0:
+                LOG.info("  near-dup scan %d/%d rows (%.1fs)", r, self.n, common.timer() - t0)
+        if not out_a:
+            return np.empty(0, dtype=np.int64), np.empty(0, dtype=np.int64)
+        a, b = np.concatenate(out_a), np.concatenate(out_b)
+        LOG.info("near-dup links: %d pairs in %.1fs", a.size, common.timer() - t0)
+        return a, b
+
+    def query(self, bases: list[str]) -> tuple[np.ndarray, np.ndarray]:
+        """Pairs (query_idx, indexed_idx) where a foreign text shares >= min_shared shingles with an indexed row.
+
+        Used to screen the (large) pretraining corpus against the (small) holdout+val reference set
+        without indexing the corpus itself: cost is one searchsorted per query row, so the millions
+        of pretrain-pretrain Enron-thread links are never materialised.
+        """
+        out_q: list[np.ndarray] = []
+        out_r: list[np.ndarray] = []
+        t0 = common.timer()
+        for qi, base in enumerate(bases):
+            h = shingle_hashes(base, self.nd)
+            if h.size == 0 or self.uniq.size == 0:
+                continue
+            g = np.searchsorted(self.uniq, h)
+            inb = g < self.uniq.size
+            g, h = g[inb], h[inb]
+            g = g[self.uniq[g] == h]          # foreign shingles absent from the index drop out here
+            if g.size == 0:
+                continue
+            cnt = self.counts[g]
+            ok = cnt <= self.nd.cap
+            if not ok.any():
+                continue
+            starts = self.start[g[ok]]
+            lens = cnt[ok]
+            idx = np.repeat(starts - np.concatenate(([0], np.cumsum(lens)[:-1])), lens) + np.arange(lens.sum())
+            u, c = np.unique(self.sorted_rid[idx], return_counts=True)
+            hit = u[c >= self.nd.min_shared]
+            if hit.size:
+                out_q.append(np.full(hit.size, qi, dtype=np.int64))
+                out_r.append(hit)
+            if qi and qi % 100_000 == 0:
+                LOG.info("  near-dup query %d/%d rows (%.1fs)", qi, len(bases), common.timer() - t0)
+        if not out_q:
+            return np.empty(0, dtype=np.int64), np.empty(0, dtype=np.int64)
+        q, r = np.concatenate(out_q), np.concatenate(out_r)
+        LOG.info("near-dup query: %d rows -> %d links in %.1fs", len(bases), q.size, common.timer() - t0)
+        return q, r
+
+
+def recount_shingle_links(
+    train_bases: list[str], val_bases: list[str], hold_bases: list[str], pretrain_bases: list[str], nd: NearDupConfig
+) -> dict[str, int]:
+    """Independent near-dup link counts between the FINAL sets, from a fresh index.
+
+    This is the leakage gate proper: it never reuses the index or masks that drove the removals,
+    so it fails if the removal logic is wrong. Same code path in `build` (after writing nothing yet)
+    and in `check` (from the parquet files). Pretrain rows are queried against a holdout+val index
+    rather than indexed, so pretrain-pretrain links are never enumerated.
+    """
+    n_tr, n_va = len(train_bases), len(val_bases)
+    index = ShingleIndex(train_bases + val_bases + hold_bases, nd)
+    la, lb = index.links()
+    part_a = np.where(la < n_tr, 0, np.where(la < n_tr + n_va, 1, 2))
+    part_b = np.where(lb < n_tr, 0, np.where(lb < n_tr + n_va, 1, 2))
+    pair = Counter(zip(np.minimum(part_a, part_b).tolist(), np.maximum(part_a, part_b).tolist()))
+    ref = ShingleIndex(hold_bases + val_bases, nd)
+    _q, r = ref.query(pretrain_bases)
+    out = {
+        "holdout_train": int(pair[(0, 2)]),
+        "holdout_val": int(pair[(1, 2)]),
+        "train_val": int(pair[(0, 1)]),
+        "holdout_pretrain": int(np.sum(r < len(hold_bases))),
+        "val_pretrain": int(np.sum(r >= len(hold_bases))),
+    }
+    LOG.info("recount shingle links (final sets): %s", out)
+    return out
+
+
+def union_find(n: int, pairs: Iterable[tuple[int, int]]) -> np.ndarray:
+    """Dense cluster ids (0..K-1, in order of first appearance) from an edge list."""
+    parent = list(range(n))
+
+    def find(x: int) -> int:
+        root = x
+        while parent[root] != root:
+            root = parent[root]
+        while parent[x] != root:
+            parent[x], x = root, parent[x]
+        return root
+
+    for a, b in pairs:
+        ra, rb = find(a), find(b)
+        if ra != rb:
+            parent[max(ra, rb)] = min(ra, rb)
+    roots = np.fromiter((find(i) for i in range(n)), dtype=np.int64, count=n)
+    _, dense = np.unique(roots, return_inverse=True)
+    return dense.astype(np.int64)
+
+
+def same_key_pairs(keys: list[str]) -> Iterable[tuple[int, int]]:
+    """Consecutive-member edges for every group of rows sharing a key."""
+    first: dict[str, int] = {}
+    for i, k in enumerate(keys):
+        j = first.setdefault(k, i)
+        if j != i:
+            yield j, i
+
+
+def same_key_pairs_multi(columns: list[list[str]]) -> Iterable[tuple[int, int]]:
+    """Edges for rows sharing a key value within OR across the given key columns.
+
+    k_pre (first 300 chars of the body) and k_sb (first 300 chars of subject+body) can collide across columns
+    when one corpus embeds the subject in the body; the leakage gate's key_union treats such a pair as shared,
+    so the clustering must too or two near-identical rows could land on different sides of a split.
+    """
+    first: dict[str, int] = {}
+    for col in columns:
+        for i, k in enumerate(col):
+            j = first.setdefault(k, i)
+            if j != i:
+                yield j, i
+
+
+# ----------------------------------------------------------------------------- loading
+def hub_sha(repo: str) -> str | None:
+    try:
+        from huggingface_hub import HfApi
+
+        return HfApi().dataset_info(repo).sha
+    except Exception as exc:  # offline or rate-limited: record unknown, keep going
+        LOG.warning("could not fetch commit sha for %s: %s", repo, exc)
+        return None
+
+
+def load_labelled(cfg: dict, limit: int | None, rng: np.random.Generator) -> tuple[pd.DataFrame, dict]:
+    """All Hub splits of the primary set concatenated; the Hub's own split is ignored (it leaks)."""
+    from datasets import concatenate_datasets, load_dataset
+
+    repo = cfg["data"]["labelled_primary"]
+    cache_dir = str(Path(cfg["paths"]["data_raw"]) / "hf_cache")
+    sha = hub_sha(repo)
+    dsd = load_dataset(repo, cache_dir=cache_dir, revision=sha)
+    split_names = list(dsd.keys())
+    parts = [dsd[s].select_columns(LABELLED_COLUMNS) for s in split_names]
+    ds = concatenate_datasets(parts)
+    df = ds.to_pandas()
+    df = df.rename(columns={"text": "body", "dataset_name": "source"})
+    df["row_idx"] = np.arange(len(df), dtype=np.int64)
+    meta = {
+        "repo": repo,
+        "commit_sha": sha,
+        "hub_splits_concatenated": {s: int(dsd[s].num_rows) for s in split_names},
+        "datasets_fingerprints": {s: dsd[s]._fingerprint for s in split_names},
+        "rows_loaded": int(len(df)),
+    }
+    LOG.info("labelled primary %s @ %s: %d rows from splits %s", repo, sha, len(df), meta["hub_splits_concatenated"])
+    if limit:
+        keep = []
+        for source, grp in df.groupby("source", sort=True):
+            n = min(limit, len(grp))
+            keep.append(grp.iloc[np.sort(rng.choice(len(grp), n, replace=False))])
+        df = pd.concat(keep).sort_values("row_idx").reset_index(drop=True)
+        LOG.warning("--limit %d: labelled subsample %d rows (per source <= %d)", limit, len(df), limit)
+    return df, meta
+
+
+def iter_pretrain_batches(cfg: dict, limit: int | None, rng: np.random.Generator, batch_size: int = 20_000):
+    """Yield (row_indices, subjects, bodies) batches of corbt without materialising the corpus."""
+    from datasets import load_dataset
+
+    repo = cfg["data"]["pretrain_corpus"]
+    cache_dir = str(Path(cfg["paths"]["data_raw"]) / "hf_cache")
+    sha = hub_sha(repo)
+    ds = load_dataset(repo, split="train", cache_dir=cache_dir, revision=sha)
+    meta = {"repo": repo, "commit_sha": sha, "datasets_fingerprint": ds._fingerprint, "rows_total": int(ds.num_rows)}
+    indices = np.arange(ds.num_rows, dtype=np.int64)
+    if limit:
+        n = min(limit * 4, ds.num_rows)
+        indices = np.sort(rng.choice(ds.num_rows, n, replace=False))
+        ds = ds.select(indices)
+        LOG.warning("--limit %d: corbt subsample %d rows", limit, n)
+    ds = ds.select_columns(PRETRAIN_COLUMNS)
+    LOG.info("pretrain corpus %s @ %s: %d rows total, processing %d", repo, sha, meta["rows_total"], len(indices))
+
+    def batches():
+        pos = 0
+        for batch in ds.iter(batch_size=batch_size):
+            n = len(batch["body"])
+            yield indices[pos : pos + n], batch["subject"], batch["body"]
+            pos += n
+
+    return meta, batches()
+
+
+# ----------------------------------------------------------------------------- policy
+def check_spam_policy(policy: str) -> None:
+    """Only `positive_class` is implementable; the others are kept so the decision stays visible."""
+    if policy == "positive_class":
+        return
+    if policy in ("map_to_zero", "exclude"):
+        raise SpamPolicyError(
+            f"spam_policy={policy!r} cannot be implemented: no dataset in scope distinguishes spam "
+            "from phishing. Every label-1 row of puyang2025/seven-phishing-email-datasets comes from "
+            "a spam-track corpus (TREC-05/06/07, CEAS-08, Enron-Spam, SpamAssassin, Ling-Spam) with a "
+            "single binary label, so mapping spam to 0 or excluding spam sources would leave ~0 "
+            "positives (verify_data.md section 5). Use positive_class (positive = phishing/spam, "
+            "'unwanted, potentially malicious') and report the credential-lure slice separately."
+        )
+    raise SpamPolicyError(f"unknown spam_policy {policy!r}; expected positive_class|map_to_zero|exclude")
+
+
+# ----------------------------------------------------------------------------- split
+def grouped_stratified_split(
+    pool: pd.DataFrame, val_fraction: float, rng: np.random.Generator
+) -> tuple[np.ndarray, dict]:
+    """Assign whole near-dup clusters to val until each source x label stratum reaches ~val_fraction.
+
+    A cluster never straddles train/val, so a template seen in training can never inflate the
+    validation score. Clusters spanning several strata are assigned under their majority stratum;
+    a cluster larger than a stratum's whole validation target goes to train.
+    """
+    strata = pool["source"].astype(str) + "|" + pool["label"].astype(str)
+    cluster = pool["cluster_id"].to_numpy()
+    members: dict[int, list[int]] = defaultdict(list)
+    for pos, cid in enumerate(cluster.tolist()):
+        members[int(cid)].append(pos)
+    stratum_arr = strata.to_numpy()
+    by_stratum: dict[str, list[tuple[int, int]]] = defaultdict(list)
+    mixed = 0
+    for cid, pos_list in members.items():
+        counts = Counter(stratum_arr[p] for p in pos_list)
+        if len(counts) > 1:
+            mixed += 1
+        by_stratum[counts.most_common(1)[0][0]].append((cid, len(pos_list)))
+
+    is_val = np.zeros(len(pool), dtype=bool)
+    per_stratum: dict[str, dict[str, Any]] = {}
+    for stratum in sorted(by_stratum):
+        groups = by_stratum[stratum]
+        n_rows = sum(size for _, size in groups)
+        target = int(round(val_fraction * n_rows))
+        order = rng.permutation(len(groups))
+        val_rows = 0
+        oversized = 0
+        for gi in order:
+            cid, size = groups[gi]
+            if size > target:
+                oversized += 1
+                continue
+            if val_rows < target and abs(val_rows + size - target) <= abs(val_rows - target):
+                for p in members[cid]:
+                    is_val[p] = True
+                val_rows += size
+        per_stratum[stratum] = {
+            "rows": n_rows, "clusters": len(groups), "val_target": target, "val_rows": val_rows,
+            "val_fraction": round(val_rows / n_rows, 4) if n_rows else 0.0,
+            "clusters_too_large_for_val": oversized,
+        }
+    info = {
+        "val_fraction_target": val_fraction,
+        "val_fraction_achieved": round(float(is_val.mean()), 4) if len(pool) else 0.0,
+        "clusters_total": len(members),
+        "clusters_mixed_stratum": mixed,
+        "largest_cluster": max((len(v) for v in members.values()), default=0),
+        "per_stratum": per_stratum,
+    }
+    return is_val, info
+
+
+# ----------------------------------------------------------------------------- bookkeeping
+def nested_counts(df: pd.DataFrame) -> dict[str, dict[str, int]]:
+    """{source: {label: n}} with string keys for JSON."""
+    out: dict[str, dict[str, int]] = {}
+    if len(df) == 0:
+        return out
+    for (source, label), n in df.groupby(["source", "label"], sort=True).size().items():
+        out.setdefault(str(source), {})[str(label)] = int(n)
+    return out
+
+
+def split_totals(df: pd.DataFrame) -> dict[str, Any]:
+    n = int(len(df))
+    pos = int((df["label"] == 1).sum()) if n else 0
+    return {"rows": n, "positives": pos, "negatives": n - pos, "base_rate": round(pos / n, 4) if n else None}
+
+
+def md_table(headers: list[str], rows: list[list[Any]]) -> str:
+    lines = ["| " + " | ".join(headers) + " |", "|" + "|".join("---" for _ in headers) + "|"]
+    for row in rows:
+        lines.append("| " + " | ".join(f"{v:,}" if isinstance(v, int) and not isinstance(v, bool) else str(v) for v in row) + " |")
+    return "\n".join(lines)
+
+
+def source_label_table(counts: dict[str, dict[str, int]]) -> str:
+    rows = []
+    for source in sorted(counts):
+        neg = counts[source].get("0", 0)
+        pos = counts[source].get("1", 0)
+        tot = neg + pos
+        rows.append([source, neg, pos, tot, f"{pos / tot:.1%}" if tot else "-"])
+    return md_table(["source", "label 0", "label 1", "rows", "positive rate"], rows)
+
+
+def peak_rss_gb() -> float:
+    return round(resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 2**20, 2)
+
+
+# ----------------------------------------------------------------------------- leakage
+@dataclass
+class LeakageResults:
+    checks: dict[str, dict[str, Any]] = field(default_factory=dict)
+
+    def add(self, name: str, count: int, assert_zero: bool, detail: str = "") -> None:
+        self.checks[name] = {"count": int(count), "asserted": assert_zero, "pass": (count == 0) or not assert_zero, "detail": detail}
+        level = logging.ERROR if (assert_zero and count) else logging.INFO
+        LOG.log(level, "leakage %-44s %8d %s", name, count, "(assert zero)" if assert_zero else "(info)")
+
+    def failures(self) -> list[str]:
+        return [k for k, v in self.checks.items() if v["asserted"] and v["count"]]
+
+    def raise_if_failed(self) -> None:
+        bad = self.failures()
+        if bad:
+            raise AssertionError("LEAKAGE ASSERTIONS FAILED: " + ", ".join(f"{k}={self.checks[k]['count']}" for k in bad))
+
+
+def leakage_checks(
+    train: pd.DataFrame,
+    val: pd.DataFrame,
+    holdout: pd.DataFrame,
+    pretrain_keys: set[str],
+    n_pretrain: int,
+    shingle_links: dict[str, int],
+    holdout_source: str,
+) -> LeakageResults:
+    """The gate. `shingle_links` must come from recount_shingle_links over the final sets and
+    `pretrain_keys` from text_identity over the final pretraining texts, in build and check alike."""
+    res = LeakageResults()
+    hold_keys = key_union(holdout)
+    val_keys = key_union(val)
+    train_keys = key_union(train)
+    res.add("holdout_vs_train_key_rows", int(rows_hitting(train, hold_keys).sum()), True)
+    res.add("holdout_vs_val_key_rows", int(rows_hitting(val, hold_keys).sum()), True)
+    res.add("holdout_vs_pretrain_key_rows", len(pretrain_keys & hold_keys), True, "pretrain keys re-derived from stored text (text_identity)")
+    res.add("holdout_vs_train_shingle_links", shingle_links["holdout_train"], True)
+    res.add("holdout_vs_val_shingle_links", shingle_links["holdout_val"], True)
+    res.add("holdout_vs_pretrain_shingle_links", shingle_links["holdout_pretrain"], True, "pretrain bases re-derived from stored text (text_identity)")
+    res.add("train_vs_val_k_full_rows", len(set(train["k_full"]) & set(val["k_full"])), True)
+    res.add("train_vs_val_any_key_rows", len(train_keys & val_keys), False)
+    res.add("train_vs_val_shingle_links", shingle_links["train_val"], False)
+    res.add("val_vs_pretrain_key_rows", len(pretrain_keys & val_keys), True)
+    res.add("val_vs_pretrain_shingle_links", shingle_links["val_pretrain"], False, "near-twins of val emails in MLM text; counted, not removed (val is model selection, the holdout is the gate)")
+    res.add("train_vs_pretrain_key_overlap", len(pretrain_keys & train_keys), False, "allowed: TREC-05/Enron ham are Enron mail")
+    res.add("holdout_rows_in_train_or_val_by_source", int((train["source"] == holdout_source).sum() + (val["source"] == holdout_source).sum()), True)
+    res.add("test_holdout_rows_from_other_sources", int((holdout["source"] != holdout_source).sum()), True)
+    res.add("pretrain_rows", n_pretrain, False)
+    return res
+
+
+# ----------------------------------------------------------------------------- build
+def cmd_build(args: argparse.Namespace, cfg: dict) -> None:
+    t_start = common.timer()
+    policy = args.spam_policy or cfg["data"]["spam_policy"]
+    check_spam_policy(policy)
+    seed = int(cfg["seed"])
+    common.set_seed(seed)
+    rng = np.random.default_rng(seed)
+    strip = not args.no_strip_quotes
+    nd = NearDupConfig.from_config(cfg, cap=args.shingle_cap or DIALS.shingle_cap)
+    holdout_source = cfg["data"]["holdout_test_source"]
+    min_alnum = int(cfg["data"]["min_alnum_chars"])
+    val_fraction = float(cfg["data"]["finetune_val_fraction"])
+    out_dir = Path(cfg["paths"]["data_processed"])
+    out_dir.mkdir(parents=True, exist_ok=True)
+    LOG.info("build: spam_policy=%s strip_quotes=%s limit=%s seed=%d holdout=%s", policy, strip, args.limit, seed, holdout_source)
+
+    # 1-2. labelled rows -> model input, clean, keys, min-alnum filter
+    df, labelled_meta = load_labelled(cfg, args.limit, rng)
+    df["subject"] = df["subject"].map(clean_text)
+    df["body"] = df["body"].map(clean_text)
+    df["text"] = [model_input(s, b) for s, b in zip(df["subject"].tolist(), df["body"].tolist())]
+    df["label"] = df["label"].astype(np.int64)
+    df["source"] = df["source"].astype(str)
+    bases = add_key_columns(df)
+    df["base"] = bases
+    LOG.info("labelled: cleaned + keyed %d rows in %.1fs", len(df), common.timer() - t_start)
+
+    short = (df["n_alnum"] < min_alnum).to_numpy()
+    drop_min_alnum = nested_counts(df.loc[short])
+    LOG.info("drop <%d alnum: %d rows %s", min_alnum, int(short.sum()), drop_min_alnum)
+    df = df.loc[~short].reset_index(drop=True)
+
+    # 3-5. holdout, Ling exclusion, exact dedup of the pool
+    is_holdout = (df["source"] == holdout_source).to_numpy()
+    is_ling = (df["source"] == "Ling").to_numpy()
+    holdout = df.loc[is_holdout].reset_index(drop=True)
+    ling_excluded = {str(k): int(v) for k, v in df.loc[is_ling].groupby("label").size().items()}
+    pool = df.loc[~is_holdout & ~is_ling].reset_index(drop=True)
+    if (pool["source"] == holdout_source).any():
+        raise AssertionError("holdout source present in the training pool")
+    LOG.info("holdout %s: %d rows (%s); Ling excluded: %d rows %s; pool before dedup: %d",
+             holdout_source, len(holdout), split_totals(holdout), int(is_ling.sum()), ling_excluded, len(pool))
+    holdout_internal_dups = int(holdout["k_full"].duplicated().sum())
+
+    # identical bodies carrying both labels (Enron ham vs TREC-05 spam energy newsletters): keeping one
+    # by Hub row order would be arbitrary label noise, so every member of a conflicting group is dropped
+    conflict = (pool.groupby("k_full")["label"].transform("nunique") > 1).to_numpy()
+    label_conflicts = {"groups": int(pool.loc[conflict, "k_full"].nunique()), "rows_dropped": int(conflict.sum()),
+                       "by_source_label": nested_counts(pool.loc[conflict])}
+    pool = pool.loc[~conflict].reset_index(drop=True)
+    LOG.info("pool label conflicts (same k_full, both labels): dropped %d rows in %d groups %s",
+             label_conflicts["rows_dropped"], label_conflicts["groups"], label_conflicts["by_source_label"])
+
+    dup_mask = pool["k_full"].duplicated(keep="first").to_numpy()
+    pool_exact_dups = nested_counts(pool.loc[dup_mask])
+    pool = pool.loc[~dup_mask].reset_index(drop=True)
+    LOG.info("pool exact dedup (k_full keep first): removed %d, kept %d", int(dup_mask.sum()), len(pool))
+
+    # 4. near-dup links over pool + holdout
+    n_pool = len(pool)
+    index = ShingleIndex(pool["base"].tolist() + holdout["base"].tolist(), nd)
+    la, lb = index.links()
+    a_hold, b_hold = la >= n_pool, lb >= n_pool
+    pool_pool = ~a_hold & ~b_hold
+    pool_hold = a_hold ^ b_hold
+    hold_hold = a_hold & b_hold
+    link_counts = {"pool_pool": int(pool_pool.sum()), "pool_holdout": int(pool_hold.sum()), "holdout_holdout": int(hold_hold.sum())}
+    n_capped, n_touching = index.n_capped_shingles, index.n_rows_touching_capped
+    LOG.info("near-dup links: %s", link_counts)
+
+    # 5-6. decontaminate the pool against the frozen holdout key/shingle sets
+    holdout_keys = frozenset(key_union(holdout))
+    key_hit = rows_hitting(pool, holdout_keys)
+    shingle_hit = np.zeros(n_pool, dtype=bool)
+    shingle_hit[np.where(b_hold[pool_hold], la[pool_hold], lb[pool_hold])] = True   # the pool side of each pool-holdout pair
+    contaminated = key_hit | shingle_hit
+    decontam = {
+        "by_source_label": nested_counts(pool.loc[contaminated]),
+        "by_reason": {"key_match_only": int((key_hit & ~shingle_hit).sum()), "shingle_link_only": int((shingle_hit & ~key_hit).sum()), "both": int((key_hit & shingle_hit).sum()), "total": int(contaminated.sum())},
+    }
+    LOG.info("decontamination vs holdout: removed %d rows %s", int(contaminated.sum()), decontam["by_reason"])
+
+    # 7. clusters: pool-pool shingle links + shared k_pre / k_sb, holdout-holdout links separately
+    edges = list(zip(la[pool_pool].tolist(), lb[pool_pool].tolist()))
+    edges += list(same_key_pairs(pool["k_pre"].tolist()))
+    edges += list(same_key_pairs(pool["k_sb"].tolist()))
+    edges += list(zip(la[hold_hold].tolist(), lb[hold_hold].tolist()))
+    cluster_all = union_find(n_pool + len(holdout), edges)
+    pool["cluster_id"] = cluster_all[:n_pool]
+    holdout["cluster_id"] = cluster_all[n_pool:]
+    pool = pool.loc[~contaminated].reset_index(drop=True)
+
+    # 9. grouped stratified train/val split
+    is_val, split_info = grouped_stratified_split(pool, val_fraction, rng)
+    train = pool.loc[~is_val].reset_index(drop=True)
+    val = pool.loc[is_val].reset_index(drop=True)
+    LOG.info("split: train %s val %s (%s)", split_totals(train), split_totals(val), {k: v for k, v in split_info.items() if k != "per_stratum"})
+    del index, la, lb
+
+    # 8. pretraining corpus: keys and near-dups vs holdout removed, keys vs val removed, near-dups vs val counted
+    val_keys = frozenset(key_union(val))
+    train_keys = frozenset(key_union(train))
+    ref_index = ShingleIndex(holdout["base"].tolist() + val["base"].tolist(), nd)
+    pretrain_df, pretrain_meta, pretrain_stats = build_pretrain(
+        cfg, args.limit, rng, strip, holdout_keys, val_keys, train_keys, ref_index, len(holdout), holdout["label"].to_numpy(),
+    )
+    del ref_index
+
+    # 11. leakage assertions: an independent recomputation over the FINAL sets, same code path as `check`
+    pretrain_keys: set[str] = set()
+    pretrain_bases: list[str] = []
+    for text in pretrain_df["text"].tolist():
+        keys, base = text_identity(text)
+        pretrain_keys |= keys
+        pretrain_bases.append(base[: nd.window_chars])
+    shingle_links = recount_shingle_links(train["base"].tolist(), val["base"].tolist(), holdout["base"].tolist(), pretrain_bases, nd)
+    del pretrain_bases
+    leak = leakage_checks(train, val, holdout, pretrain_keys, len(pretrain_df), shingle_links, holdout_source)
+    leak.raise_if_failed()
+
+    # 10. write outputs
+    files = write_outputs(out_dir, train, val, holdout, pretrain_df)
+    elapsed = round(common.timer() - t_start, 1)
+    manifest = {
+        "timestamp": datetime.now(timezone.utc).isoformat(timespec="seconds"),
+        "limit": args.limit,
+        "dev_subsample": bool(args.limit),
+        "spam_policy": policy,
+        "strip_quotes": strip,
+        "seed": seed,
+        "holdout_test_source": holdout_source,
+        "elapsed_seconds": elapsed,
+        "peak_rss_gb": peak_rss_gb(),
+        "datasets": {
+            "labelled_primary": labelled_meta,
+            "pretrain_corpus": pretrain_meta,
+            "excluded": cfg["data"]["labelled_excluded"],
+        },
+        "counts": {
+            "splits": {"finetune_train": nested_counts(train), "finetune_val": nested_counts(val), "test_holdout": nested_counts(holdout)},
+            "totals": {"finetune_train": split_totals(train), "finetune_val": split_totals(val), "test_holdout": split_totals(holdout), "pretrain": {"rows": int(len(pretrain_df)), "chars": int(pretrain_df["n_chars"].sum())}},
+        },
+        "normalization": "common.normalize_text: NFKC -> lowercase -> keep [a-z0-9] only (import it, never re-derive it; NFKC is an extension of the verify_data.md section 7 wording)",
+        "dials": dict(DIALS.__dict__),
+        "drops": {
+            "labelled_min_alnum": {"threshold": min_alnum, "rows": int(short.sum()), "by_source_label": drop_min_alnum},
+            "ling_excluded_by_label": ling_excluded,
+            "pool_label_conflicts": label_conflicts,
+            "pool_exact_dups_by_source_label": pool_exact_dups,
+            "decontamination_vs_holdout": decontam,
+        },
+        "holdout": {"rows": int(len(holdout)), "internal_exact_dups_kept": holdout_internal_dups},
+        "near_dup": {**nd.__dict__, "capped_shingles": n_capped, "rows_touching_capped": n_touching, "links_at_removal_stage": link_counts},
+        "split": split_info,
+        "pretrain": pretrain_stats,
+        "leakage": leak.checks,
+        "files": files,
+        "config_snapshot": {k: v for k, v in cfg.items() if not k.startswith("_")},
+    }
+    common.write_json(manifest, out_dir / "splits_manifest.json")
+    (out_dir / "data_report.md").write_text(render_report(manifest), encoding="utf-8")
+    LOG.info("wrote %s and %s in %.1fs (peak RSS %.2f GB)", out_dir / "splits_manifest.json", out_dir / "data_report.md", elapsed, manifest["peak_rss_gb"])
+    print_summary(manifest)
+
+
+def build_pretrain(
+    cfg: dict, limit: int | None, rng: np.random.Generator, strip: bool,
+    holdout_keys: frozenset[str], val_keys: frozenset[str], train_keys: frozenset[str],
+    ref_index: ShingleIndex, n_holdout: int, holdout_labels: np.ndarray,
+) -> tuple[pd.DataFrame, dict, dict]:
+    """corbt -> deduped, quote-stripped MLM corpus with no exact or near-duplicate of a holdout email.
+
+    Streaming pass: exact dedup, quote strip, length filter, key screens against holdout and val.
+    Then one near-dup query of every surviving row against `ref_index` (holdout bases first, then
+    val bases): rows linked to a holdout row are removed (verify_data.md section 7 step 8, which
+    the key screens alone miss for quoted/forwarded variants and 419 templates), rows linked to a
+    val row are counted but kept. Empty-subject rows are additionally screened under their
+    text-derived identity (split_text) so the post-hoc text-only assertion shared with `check` can
+    never see a key or link this pass did not.
+    """
+    meta, batches = iter_pretrain_batches(cfg, limit, rng)
+    seen_raw: set[str] = set()
+    seen_stripped: set[str] = set()
+    doc_ids: list[str] = []
+    texts: list[str] = []
+    n_chars: list[int] = []
+    q_bases: list[str] = []      # shingle bases to screen (one per row, two for empty-subject rows)
+    q_row: list[int] = []        # kept-row index of each q_base
+    window = ref_index.nd.window_chars
+    st = Counter()
+    marker_hits = Counter()
+    chars_before = chars_after = chars_cut_bodies_before = chars_cut_bodies_after = 0
+    t0 = common.timer()
+    for idx_batch, subjects, bodies in batches:
+        for row_idx, subj, body in zip(idx_batch.tolist(), subjects, bodies):
+            st["rows_in"] += 1
+            subj = clean_text(subj)
+            body = clean_text(body)
+            raw_key = common.sha1_hex(common.normalize_text(subj) + common.normalize_text(body))
+            if raw_key in seen_raw:
+                st["exact_dups_raw"] += 1
+                continue
+            seen_raw.add(raw_key)
+            chars_before += len(body)
+            if strip:
+                new_body, marker, cut = strip_quotes(body)
+                if marker:
+                    marker_hits[marker] += 1
+                if cut:
+                    st["bodies_cut"] += 1
+                    chars_cut_bodies_before += len(body)
+                    chars_cut_bodies_after += len(new_body)
+                elif marker:
+                    st["marker_but_kept_full"] += 1
+                body = new_body
+            chars_after += len(body)
+            k_full, k_pre, k_sb, k_sb_full, base, _n = row_keys(subj, body)
+            if len(common.normalize_text(body)) < DIALS.pretrain_min_alnum:
+                st["short_after_strip"] += 1
+                continue
+            if k_sb_full in seen_stripped:
+                st["exact_dups_after_strip"] += 1
+                continue
+            seen_stripped.add(k_sb_full)
+            keys = {k_full, k_pre, k_sb, k_sb_full}
+            text = model_input(subj, body)
+            alt_base: str | None = None
+            if not subj:                      # `check` will see (first paragraph, rest): screen that identity too
+                d_keys, d_base = text_identity(text)
+                keys |= d_keys
+                if d_base != base:
+                    alt_base = d_base
+            if keys & holdout_keys:
+                st["holdout_key_removed"] += 1
+                continue
+            if keys & val_keys:
+                st["val_key_removed"] += 1
+                continue
+            if keys & train_keys:
+                st["train_overlap_kept"] += 1
+            q_bases.append(base[:window])
+            q_row.append(len(texts))
+            if alt_base is not None:
+                q_bases.append(alt_base[:window])
+                q_row.append(len(texts))
+            doc_ids.append(f"corbt:{row_idx:06d}")
+            texts.append(text)
+            n_chars.append(len(text))
+        if st["rows_in"] % 100_000 < 20_000:
+            LOG.info("  corbt %d rows in, %d kept (%.1fs)", st["rows_in"], len(texts), common.timer() - t0)
+
+    # near-dup screen against the frozen holdout (removed) and val (counted) shingle sets
+    q, r = ref_index.query(q_bases)
+    del q_bases
+    row_of = np.asarray(q_row, dtype=np.int64)[q] if q.size else np.empty(0, dtype=np.int64)
+    hit_hold = r < n_holdout
+    drop = np.zeros(len(texts), dtype=bool)
+    drop[row_of[hit_hold]] = True
+    val_linked_rows = np.unique(row_of[~hit_hold & ~drop[row_of]])
+    hold_rows_hit = np.unique(r[hit_hold])
+    near_dup = {
+        "holdout_links": int(hit_hold.sum()),
+        "holdout_rows_removed": int(drop.sum()),
+        "holdout_rows_hit_by_label": {str(k): int(v) for k, v in Counter(holdout_labels[hold_rows_hit].tolist()).items()},
+        "val_links": int((~hit_hold).sum()),
+        "val_rows_hit": int(np.unique(r[~hit_hold] - n_holdout).size),
+        "pretrain_rows_near_val_kept": int(val_linked_rows.size),
+    }
+    LOG.info("pretrain near-dup screen: %s", near_dup)
+    keep = ~drop
+    df = pd.DataFrame({
+        "doc_id": [d for d, k in zip(doc_ids, keep) if k],
+        "text": [t for t, k in zip(texts, keep) if k],
+        "n_chars": np.asarray(n_chars, dtype=np.int64)[keep],
+    })
+    stats = {
+        **{k: int(st[k]) for k in ("rows_in", "exact_dups_raw", "short_after_strip", "exact_dups_after_strip", "holdout_key_removed", "val_key_removed", "train_overlap_kept")},
+        "holdout_near_dup_removed": near_dup["holdout_rows_removed"],
+        "near_dup_screen": near_dup,
+        "rows_out": int(len(df)),
+        "total_chars_out": int(df["n_chars"].sum()) if len(df) else 0,
+        "mean_chars_out": round(float(df["n_chars"].mean()), 1) if len(df) else 0.0,
+        "min_alnum_after_strip": DIALS.pretrain_min_alnum,
+        "quote_strip": {
+            "enabled": strip,
+            "bodies_cut": int(st["bodies_cut"]),
+            "marker_present_but_kept_full": int(st["marker_but_kept_full"]),
+            "marker_hits": dict(marker_hits),
+            "retained_char_fraction_all_bodies": round(chars_after / chars_before, 4) if chars_before else None,
+            "retained_char_fraction_cut_bodies": round(chars_cut_bodies_after / chars_cut_bodies_before, 4) if chars_cut_bodies_before else None,
+            "min_alnum_above_cut": DIALS.quote_strip_min_alnum,
+        },
+    }
+    LOG.info("pretrain corpus: %s", {k: v for k, v in stats.items() if k not in ("quote_strip", "near_dup_screen")})
+    LOG.info("quote strip: %s", stats["quote_strip"])
+    return df, meta, stats
+
+
+def write_outputs(out_dir: Path, train: pd.DataFrame, val: pd.DataFrame, holdout: pd.DataFrame, pretrain: pd.DataFrame) -> dict[str, str]:
+    files = {}
+    for name, df in (("finetune_train", train), ("finetune_val", val), ("test_holdout", holdout)):
+        out = df.copy()
+        out["doc_id"] = [f"primary:{i:06d}" for i in out["row_idx"].tolist()]
+        out = out[FINETUNE_SCHEMA]
+        out["label"] = out["label"].astype(np.int64)
+        out["cluster_id"] = out["cluster_id"].astype(np.int64)
+        path = out_dir / f"{name}.parquet"
+        out.to_parquet(path, index=False)
+        files[name] = str(path)
+        LOG.info("wrote %s (%d rows)", path, len(out))
+    path = out_dir / "pretrain.parquet"
+    pretrain[PRETRAIN_SCHEMA].to_parquet(path, index=False)
+    files["pretrain"] = str(path)
+    LOG.info("wrote %s (%d rows)", path, len(pretrain))
+    return files
+
+
+# ----------------------------------------------------------------------------- report
+def render_report(m: dict) -> str:
+    c = m["counts"]
+    tot = c["totals"]
+    lines = [f"# Data report — {m['timestamp']}", ""]
+    if m["dev_subsample"]:
+        lines += [f"**DEV SUBSAMPLE (--limit {m['limit']}): counts below are NOT the real build.**", ""]
+    lines += [
+        f"spam_policy `{m['spam_policy']}` (positive = phishing/spam, unwanted); quote stripping {'on' if m['strip_quotes'] else 'off'}; "
+        f"seed {m['seed']}; holdout source `{m['holdout_test_source']}`; elapsed {m['elapsed_seconds']}s; peak RSS {m['peak_rss_gb']} GB.",
+        "",
+        "## Sources",
+        f"- labelled primary: `{m['datasets']['labelled_primary']['repo']}` @ `{m['datasets']['labelled_primary']['commit_sha']}` "
+        f"({m['datasets']['labelled_primary']['rows_loaded']:,} rows, hub splits {m['datasets']['labelled_primary']['hub_splits_concatenated']} concatenated; the hub split leaks and is ignored)",
+        f"- pretraining corpus: `{m['datasets']['pretrain_corpus']['repo']}` @ `{m['datasets']['pretrain_corpus']['commit_sha']}` ({m['datasets']['pretrain_corpus']['rows_total']:,} rows)",
+        f"- excluded (no body text / 92% subset): {', '.join('`' + r + '`' for r in m['datasets']['excluded'])}",
+        "",
+        "## Splits",
+        md_table(["split", "rows", "positives", "negatives", "base rate"],
+                 [[k, v["rows"], v["positives"], v["negatives"], v["base_rate"]] for k, v in tot.items() if k != "pretrain"]
+                 + [["pretrain (MLM)", tot["pretrain"]["rows"], "-", "-", f"{tot['pretrain']['chars']:,} chars"]]),
+        "",
+    ]
+    for split in ("finetune_train", "finetune_val", "test_holdout"):
+        lines += [f"### {split} by source", source_label_table(c["splits"][split]), ""]
+    d = m["drops"]
+    lc = d["pool_label_conflicts"]
+    lines += [
+        "## Normalisation and dials",
+        f"- {m['normalization']}",
+        f"- dials: {m['dials']}",
+        "",
+        "## Drops and removals",
+        f"- labelled rows with <{d['labelled_min_alnum']['threshold']} alphanumerics (subject+body): {d['labelled_min_alnum']['rows']:,} rows",
+        source_label_table(d["labelled_min_alnum"]["by_source_label"]),
+        f"- Ling excluded (academic list corpus, {sum(d['ling_excluded_by_label'].values()):,} rows by label {d['ling_excluded_by_label']})",
+        f"- pool label conflicts (identical normalised body labelled both 0 and 1; all members dropped): {lc['rows_dropped']:,} rows in {lc['groups']:,} groups",
+        source_label_table(lc["by_source_label"]),
+        f"- pool exact duplicates (k_full, keep first): {sum(sum(v.values()) for v in d['pool_exact_dups_by_source_label'].values()):,} rows",
+        source_label_table(d["pool_exact_dups_by_source_label"]),
+        f"- decontamination against the holdout: {d['decontamination_vs_holdout']['by_reason']}",
+        source_label_table(d["decontamination_vs_holdout"]["by_source_label"]),
+        f"- holdout internal exact duplicates kept (test set is never deduped): {m['holdout']['internal_exact_dups_kept']:,}",
+        "",
+        "## Near-duplicate clustering",
+        f"- {m['near_dup']['shingle_chars']}-char shingles, stride {m['near_dup']['stride']}, first {m['near_dup']['window_chars']} chars, link at >= {m['near_dup']['min_shared']} shared; "
+        f"{m['near_dup']['capped_shingles']:,} boilerplate shingles over cap {m['near_dup']['cap']:,} ignored (touching {m['near_dup']['rows_touching_capped']:,} rows)",
+        f"- links at the removal stage (pool + holdout index): {m['near_dup']['links_at_removal_stage']}",
+        f"- clusters {m['split']['clusters_total']:,} (mixed source x label: {m['split']['clusters_mixed_stratum']:,}; largest {m['split']['largest_cluster']:,}); "
+        f"val fraction target {m['split']['val_fraction_target']} achieved {m['split']['val_fraction_achieved']}",
+        md_table(["stratum", "rows", "clusters", "val target", "val rows", "val frac", "clusters too large"],
+                 [[k, v["rows"], v["clusters"], v["val_target"], v["val_rows"], v["val_fraction"], v["clusters_too_large_for_val"]] for k, v in m["split"]["per_stratum"].items()]),
+        "",
+        "## Pretraining corpus (corbt/enron-emails)",
+    ]
+    p = m["pretrain"]
+    lines += [md_table(["stage", "rows"], [
+        ["rows in", p["rows_in"]], ["exact dups (subject+body, raw)", p["exact_dups_raw"]],
+        [f"< {p['min_alnum_after_strip']} alnum after stripping", p["short_after_strip"]],
+        ["exact dups after stripping", p["exact_dups_after_strip"]],
+        ["matches a holdout key (removed)", p["holdout_key_removed"]], ["matches a fine-tune val key (removed)", p["val_key_removed"]],
+        [f">= {m['near_dup']['min_shared']} shingles shared with a holdout row (removed)", p["holdout_near_dup_removed"]],
+        [f">= {m['near_dup']['min_shared']} shingles shared with a fine-tune val row (kept, counted)", p["near_dup_screen"]["pretrain_rows_near_val_kept"]],
+        ["overlaps a fine-tune train key (kept, allowed)", p["train_overlap_kept"]], ["rows out", p["rows_out"]],
+    ])]
+    ns = p["near_dup_screen"]
+    q = p["quote_strip"]
+    lines += [
+        f"- near-dup screen: {ns['holdout_links']:,} links to {sum(ns['holdout_rows_hit_by_label'].values()):,} holdout rows (by label {ns['holdout_rows_hit_by_label']}); "
+        f"{ns['val_links']:,} links to {ns['val_rows_hit']:,} val rows",
+        f"- quote/forward stripping {'on' if q['enabled'] else 'off'}: {q['bodies_cut']:,} bodies cut, {q['marker_present_but_kept_full']:,} had a marker but kept the full body "
+        f"(< {q['min_alnum_above_cut']} alnum above it); marker hits {q['marker_hits']}; retained chars {q['retained_char_fraction_all_bodies']} of all bodies, "
+        f"{q['retained_char_fraction_cut_bodies']} of cut bodies",
+        f"- {p['total_chars_out']:,} chars out, mean {p['mean_chars_out']} chars/doc",
+        "",
+        "## Leakage assertions",
+        md_table(["check", "count", "asserted zero", "pass"], [[k, v["count"], v["asserted"], v["pass"]] for k, v in m["leakage"].items()]),
+        "",
+        "## Files",
+    ] + [f"- `{k}`: `{v}`" for k, v in m["files"].items()]
+    return "\n".join(lines) + "\n"
+
+
+def print_summary(m: dict) -> None:
+    print("\n=== data_prep summary ===")
+    if m["dev_subsample"]:
+        print(f"*** DEV SUBSAMPLE --limit {m['limit']} — not a real build ***")
+    for k, v in m["counts"]["totals"].items():
+        print(f"{k:>16}: {v}")
+    print("leakage:")
+    for k, v in m["leakage"].items():
+        print(f"  {k:<44} {v['count']:>8}  {'ASSERT' if v['asserted'] else 'info  '}  {'ok' if v['pass'] else 'FAIL'}")
+    print(f"pretrain: {m['pretrain']['rows_out']:,} rows, quote strip {m['pretrain']['quote_strip']['bodies_cut']:,} cut")
+
+
+# ----------------------------------------------------------------------------- check
+def cmd_check(args: argparse.Namespace, cfg: dict) -> None:
+    out_dir = Path(cfg["paths"]["data_processed"])
+    manifest = common.read_json(out_dir / "splits_manifest.json")
+    nd = NearDupConfig.from_config(cfg, cap=int(manifest["near_dup"]["cap"]))
+    frames = {}
+    for name in ("finetune_train", "finetune_val", "test_holdout"):
+        df = pd.read_parquet(out_dir / f"{name}.parquet")
+        if list(df.columns) not in (FINETUNE_SCHEMA, THREEWAY_SCHEMA):   # build3 appends label3/label_set; the binary gate ignores them
+            raise AssertionError(f"{name}: unexpected columns {list(df.columns)}")
+        df = df[FINETUNE_SCHEMA]
+        df["subject"] = df["subject"].fillna("")
+        df["body"] = df["body"].fillna("")
+        df["base"] = add_key_columns(df)
+        frames[name] = df
+        LOG.info("read %s: %d rows", name, len(df))
+    pre = pd.read_parquet(out_dir / "pretrain.parquet", columns=["text"])
+    if list(pre.columns) != ["text"]:
+        raise AssertionError("pretrain.parquet: missing text column")
+    pretrain_keys: set[str] = set()
+    pretrain_bases: list[str] = []
+    for t in pre["text"].tolist():
+        keys, base = text_identity(t)
+        pretrain_keys |= keys
+        pretrain_bases.append(base[: nd.window_chars])
+    LOG.info("read pretrain: %d rows", len(pre))
+
+    train, val, hold = frames["finetune_train"], frames["finetune_val"], frames["test_holdout"]
+    holdout_source = cfg["data"]["holdout_test_source"]
+    shingle_links = recount_shingle_links(train["base"].tolist(), val["base"].tolist(), hold["base"].tolist(), pretrain_bases, nd)
+    leak = leakage_checks(train, val, hold, pretrain_keys, len(pre), shingle_links, holdout_source)
+    print_summary({**manifest, "leakage": leak.checks})
+    print("\nmanifest timestamp:", manifest["timestamp"], "| limit:", manifest["limit"], "| spam_policy:", manifest["spam_policy"])
+    print("build-time leakage results:", {k: v["count"] for k, v in manifest["leakage"].items()})
+    leak.raise_if_failed()
+    print("ALL LEAKAGE ASSERTIONS PASS")
+
+
+# ----------------------------------------------------------------------------- inspect
+def cmd_inspect(args: argparse.Namespace, cfg: dict) -> None:
+    from datasets import get_dataset_split_names, load_dataset
+
+    cache_dir = str(Path(cfg["paths"]["data_raw"]) / "hf_cache")
+    repos = [cfg["data"]["labelled_primary"], cfg["data"]["pretrain_corpus"], *cfg["data"]["labelled_excluded"]]
+    for repo in repos:
+        print(f"\n=== {repo} ===")
+        sha = hub_sha(repo)
+        try:
+            splits = get_dataset_split_names(repo, revision=sha)
+            ds = load_dataset(repo, split=splits[0], streaming=True, cache_dir=cache_dir, revision=sha)
+            print(f"commit: {sha}  splits: {splits}  (showing '{splits[0]}')")
+            rows = list(ds.take(2))
+            if ds.features is not None:
+                print("features:")
+                for name, feat in ds.features.items():
+                    print(f"  {name}: {feat}")
+            else:   # streaming CSV/JSON builders declare no schema; show the python types of the first row
+                print("features: not declared by the builder; python types of row 0:")
+                for name, value in (rows[0].items() if rows else []):
+                    print(f"  {name}: {type(value).__name__}")
+            for i, row in enumerate(rows):
+                print(f"row {i}:")
+                for k, v in row.items():
+                    s = repr(v)
+                    print(f"  {k}: {s[:200]}{'...' if len(s) > 200 else ''}")
+        except Exception as exc:
+            print(f"ERROR inspecting {repo}: {type(exc).__name__}: {exc}")
+    role = {cfg["data"]["labelled_primary"]: "labelled primary (own split built here)", cfg["data"]["pretrain_corpus"]: "MLM pretraining"}
+    print("\nroles:", {r: role.get(r, "EXCLUDED from training/eval") for r in repos})
+
+
+# ----------------------------------------------------------------------------- three-way labels (RUN_LOG entry 6)
+# Contract shared with finetune.py / evaluate.py / app: label3 in {0 legitimate, 1 phishing, 2 spam, -1 unwanted-but-
+# unsplit}, label_set in {legit, phish, spam, unwanted}; the binary `label` stays (1 for phish/spam/unwanted).
+LABEL3_OF_SET: dict[str, int] = {"legit": 0, "phish": 1, "spam": 2, "unwanted": -1}
+SET_OF_LABEL3: dict[int, str] = {v: k for k, v in LABEL3_OF_SET.items()}
+LABEL3_NAMES: dict[int, str] = {0: "Legitimate", 1: "Phishing", 2: "Spam"}
+UNWANTED_NAME = "Unwanted (phishing or spam)"
+THREEWAY_SCHEMA = FINETUNE_SCHEMA + ["label3", "label_set"]
+THREEWAY_SPLITS = ("finetune_train", "finetune_val", "test_holdout", "test_holdout3")
+BINARY_SNAPSHOT_DIR = "binary_snapshot"
+BIGSHP_SOURCE = "bigshp"
+PART_ORDER = ("holdout", "test3", "train", "val")   # canonical order of the shingle-link pair names (a_b with a before b)
+MBOX_INTERNAL_SUBJECT = "DON'T DELETE THIS MESSAGE -- FOLDER INTERNAL DATA"   # mbox folder header mis-exported as an email
+_WORD = re.compile(r"[a-z]{2,}")
+_PUNCT = re.compile(r"[^\w\s]")
+
+THREEWAY_DEFAULTS: dict[str, Any] = {
+    # Provenance -> label3 of a binary-positive row. TREC/CEAS spam tracks never separated spam from phishing
+    # ("unwanted", -1, trains on P(phish)+P(spam)); SpamAssassin / Enron-Spam / Ling label-1 rows are spam folders;
+    # Nazario and Nigerian_Fraud are phishing corpora by construction.
+    "positive_label3_by_source": {
+        "Assassin": 2, "Enron": 2, "Ling": 2,
+        "TREC-05": -1, "TREC-06": -1, "TREC-07": -1, "CEAS-08": -1,
+        "Nazario": 1, "Nigerian_Fraud": 1,
+    },
+    "champa": {
+        "repo": "kudzaiprichard/aura-phishing-email-corpus",
+        "files": ["Nazario.csv", "Nigerian_Fraud.csv"],
+        "local_dir": "data/raw/aura",
+        "label3": 1,
+    },
+    "bigshp": {
+        "enabled": False,
+        "verdict": "DO_NOT_USE",
+        "repo": "locuoco/the-biggest-spam-ham-phish-email-dataset-300000",
+        "file": "df.csv",
+        "local_dir": "data/raw/bigshp",
+        "drop_phish_class": True,
+        "min_chars": 160,
+        "min_english_fraction": 0.7,
+        "near_dup_token_key_len": 30,
+    },
+    "new_source_test_fraction": 0.15,
+    "new_source_val_fraction": 0.10,
+    "drop_new_clusters_linked_to_both_train_and_val": True,
+    "holdout3_from_val": {
+        "legit_sources": ["CEAS-08", "TREC-07", "Assassin"],
+        "spam_sources": ["Assassin", "Enron"],
+        "legit_rows": 2000,
+        "spam_rows": 700,
+    },
+}
+
+
+def threeway_config(cfg: dict) -> dict[str, Any]:
+    """config `data.threeway` merged over THREEWAY_DEFAULTS one level deep.
+
+    Every decision (which source is spam vs unwanted, whether bigshp is ingested, the holdout3 targets) is
+    therefore visible in config.yaml and reversible, while a missing key never silently changes a decision.
+    """
+    user = (cfg.get("data") or {}).get("threeway") or {}
+    out: dict[str, Any] = {}
+    for key, default in THREEWAY_DEFAULTS.items():
+        val = user.get(key, default)
+        if isinstance(default, dict) and isinstance(val, dict):
+            merged = dict(default)
+            merged.update(val)
+            val = merged
+        out[key] = val
+    return out
+
+
+def label3_from_provenance(source: np.ndarray, label: np.ndarray, pos_map: dict[str, int]) -> np.ndarray:
+    """label3 of every row from (source, binary label): label 0 -> 0, label 1 -> pos_map[source].
+
+    Raises on a positive row whose source has no entry: silently defaulting it to "unwanted" would hide a
+    provenance decision that belongs in config.yaml.
+    """
+    label = np.asarray(label, dtype=np.int64)
+    source = np.asarray(source).astype(str)
+    pos = label == 1
+    unknown = sorted(set(source[pos].tolist()) - set(pos_map))
+    if unknown:
+        raise ValueError(f"no data.threeway.positive_label3_by_source entry for positive rows of source(s) {unknown}")
+    out = np.zeros(len(label), dtype=np.int64)
+    for s in set(source[pos].tolist()):
+        out[pos & (source == s)] = int(pos_map[s])
+    return out
+
+
+def add_label3_columns(df: pd.DataFrame, label3: np.ndarray) -> None:
+    df["label3"] = np.asarray(label3, dtype=np.int64)
+    df["label_set"] = [SET_OF_LABEL3[int(v)] for v in df["label3"].tolist()]
+
+
+def contract_violations(df: pd.DataFrame, pos_map: dict[str, int]) -> dict[str, int]:
+    """Per-rule counts of rows breaking the three-way contract (all zero on a correct file).
+
+    label_set must be a known set, label3 must equal LABEL3_OF_SET[label_set], the binary label must be
+    (label3 != 0), and for every source except bigshp (which carries its own label) label3 must equal the
+    provenance mapping. check3 asserts the sum is zero.
+    """
+    if len(df) == 0:
+        return {"label_set_unknown": 0, "label3_vs_label_set": 0, "label_vs_label3": 0, "label3_vs_provenance": 0}
+    label3 = df["label3"].to_numpy().astype(np.int64)
+    lset = df["label_set"].astype(str).to_numpy()
+    label = df["label"].to_numpy().astype(np.int64)
+    expect3 = np.fromiter((LABEL3_OF_SET.get(s, -99) for s in lset.tolist()), dtype=np.int64, count=len(lset))
+    out = {
+        "label_set_unknown": int((expect3 == -99).sum()),
+        "label3_vs_label_set": int((expect3 != label3).sum()),
+        "label_vs_label3": int(((label3 != 0).astype(np.int64) != label).sum()),
+    }
+    own = df["source"].astype(str).to_numpy() == BIGSHP_SOURCE
+    try:
+        prov = label3_from_provenance(df["source"].to_numpy()[~own], label[~own], pos_map)
+        out["label3_vs_provenance"] = int((prov != label3[~own]).sum())
+    except ValueError as exc:
+        LOG.error("provenance mapping: %s", exc)
+        out["label3_vs_provenance"] = int((~own).sum())
+    out["bigshp_label3_outside_0_1_2"] = int((~np.isin(label3[own], [0, 1, 2])).sum())
+    return out
+
+
+def nested_counts3(df: pd.DataFrame) -> dict[str, dict[str, int]]:
+    """{source: {label3: n}} with string keys for JSON."""
+    out: dict[str, dict[str, int]] = {}
+    if len(df) == 0:
+        return out
+    for (source, l3), n in df.groupby(["source", "label3"], sort=True).size().items():
+        out.setdefault(str(source), {})[str(int(l3))] = int(n)
+    return out
+
+
+def split_totals3(df: pd.DataFrame) -> dict[str, Any]:
+    n = int(len(df))
+    by3 = {str(k): int(v) for k, v in df["label3"].value_counts().sort_index().items()} if n else {}
+    byset = {str(k): int(v) for k, v in df["label_set"].value_counts().sort_index().items()} if n else {}
+    pos = int((df["label"] == 1).sum()) if n else 0
+    return {"rows": n, "by_label3": by3, "by_label_set": byset, "binary_positives": pos,
+            "binary_base_rate": round(pos / n, 4) if n else None,
+            "rows_with_full_label3": int(df["label3"].isin([0, 1, 2]).sum()) if n else 0}
+
+
+def source_label3_table(counts: dict[str, dict[str, int]]) -> str:
+    rows = []
+    for source in sorted(counts):
+        c = counts[source]
+        rows.append([source, c.get("0", 0), c.get("1", 0), c.get("2", 0), c.get("-1", 0), sum(c.values())])
+    return md_table(["source", "0 Legitimate", "1 Phishing", "2 Spam", "-1 unwanted (unsplit)", "rows"], rows)
+
+
+def first_reason(n: int, reasons: list[tuple[str, np.ndarray]]) -> np.ndarray:
+    """Per-row first matching drop reason ("" = kept), so every drop is counted exactly once."""
+    out = np.full(n, "", dtype=object)
+    for name, mask in reasons:
+        hit = np.asarray(mask, dtype=bool) & (out == "")
+        out[hit] = name
+    return out
+
+
+def by_source_reason(source: np.ndarray, reason: np.ndarray) -> dict[str, dict[str, int]]:
+    out: dict[str, dict[str, int]] = {}
+    for (s, r), n in Counter(zip(source.tolist(), reason.tolist())).items():
+        if r:
+            out.setdefault(str(s), {})[str(r)] = int(n)
+    return out
+
+
+def write_table_atomic(table: "pa.Table", path: Path) -> None:
+    """tmp + os.replace: other tools (finetune, evaluate, tokenizer stats) read these files while we write."""
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_name(path.name + ".tmp")
+    pq.write_table(table, tmp)
+    os.replace(tmp, path)
+
+
+def write_text_atomic(text: str, path: Path) -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_name(path.name + ".tmp")
+    tmp.write_text(text, encoding="utf-8")
+    os.replace(tmp, path)
+
+
+def copy_atomic(src: Path, dst: Path) -> None:
+    dst.parent.mkdir(parents=True, exist_ok=True)
+    tmp = dst.with_name(dst.name + ".tmp")
+    shutil.copy2(src, tmp)
+    os.replace(tmp, dst)
+
+
+# ----------------------------------------------------------------------------- three-way: inputs
+def binary_base_dir(processed_dir: Path, dev: bool) -> tuple[Path, str]:
+    """Directory holding the BINARY (pre-build3) splits that build3 derives from.
+
+    build3 rewrites finetune_{train,val}.parquet and test_holdout.parquet in place, so a second run could not
+    start from the same inputs. The first non-dev run therefore snapshots the binary files into
+    <data_processed>/binary_snapshot/ (atomic copies) and every later run derives from that snapshot: build3 is
+    idempotent and the binary split membership stays byte-for-byte recoverable. If the live files already carry
+    label3 and there is no snapshot, the binary originals are gone and `build` must be re-run first.
+    """
+    names = ("finetune_train", "finetune_val", "test_holdout")
+    snap = processed_dir / BINARY_SNAPSHOT_DIR
+    if all((snap / f"{n}.parquet").exists() for n in names):
+        return snap, "snapshot"
+    for n in names:
+        live = processed_dir / f"{n}.parquet"
+        if not live.exists():
+            raise FileNotFoundError(f"{live} missing: run `data_prep.py build` first")
+        if "label3" in pq.read_schema(live).names:
+            raise RuntimeError(f"{live} already carries label3 but {snap} has no snapshot of the binary files; "
+                               "re-run `data_prep.py build` to recreate the binary splits before build3")
+    if dev:
+        return processed_dir, "live (dev run: no snapshot taken)"
+    for n in names:
+        copy_atomic(processed_dir / f"{n}.parquet", snap / f"{n}.parquet")
+    LOG.info("snapshotted the binary splits to %s", snap)
+    return snap, "snapshot (created by this run)"
+
+
+def per_source_sample(source: np.ndarray, limit: int, rng: np.random.Generator) -> np.ndarray:
+    """Sorted row indices: at most `limit` rows per source (dev subsample)."""
+    keep = []
+    for s in np.unique(source):
+        idx = np.flatnonzero(source == s)
+        keep.append(rng.choice(idx, min(limit, idx.size), replace=False))
+    return np.sort(np.concatenate(keep)) if keep else np.empty(0, dtype=np.int64)
+
+
+def read_binary_split(path: Path, limit: int | None, rng: np.random.Generator) -> tuple["pa.Table", pd.DataFrame]:
+    """(arrow table, keyed frame) of a binary split; the two stay row-aligned so the table can be rewritten in place."""
+    table = pq.read_table(path)
+    if list(table.schema.names) != FINETUNE_SCHEMA:
+        raise AssertionError(f"{path}: expected binary columns {FINETUNE_SCHEMA}, got {list(table.schema.names)}")
+    if limit:
+        idx = per_source_sample(np.asarray(table.column("source").to_pylist(), dtype=object), limit, rng)
+        table = table.take(pa.array(idx))
+    df = table.select(["doc_id", "subject", "body", "label", "source", "cluster_id"]).to_pandas()
+    df["subject"] = df["subject"].fillna("")
+    df["body"] = df["body"].fillna("")
+    df["label"] = df["label"].astype(np.int64)
+    df["source"] = df["source"].astype(str)
+    df["base"] = add_key_columns(df)
+    LOG.info("read %s: %d rows%s", path.name, len(df), f" (dev subsample, <= {limit}/source)" if limit else "")
+    return table, df
+
+
+def pretrain_identity(path: Path, nd: NearDupConfig, limit: int | None, rng: np.random.Generator
+                      ) -> tuple[set[str], list[str], int, "pa.Table | None"]:
+    """(keys, window-trimmed shingle bases, n_rows, dev table) of the pretraining corpus.
+
+    Re-derived from the stored `text` with text_identity exactly as `check` does, in 20k-row batches so the
+    408M-char corpus is never fully resident. In dev mode a 4*limit subsample is used AND returned so the dev
+    directory gets its own pretrain.parquet: check3 must screen against the same corpus build3 screened against.
+    """
+    keys: set[str] = set()
+    bases: list[str] = []
+    dev_table = None
+    if limit:
+        full = pq.read_table(path)
+        n = min(limit * 4, full.num_rows)
+        idx = np.sort(rng.choice(full.num_rows, n, replace=False))
+        dev_table = full.take(pa.array(idx))
+        del full
+        batches = dev_table.select(["text"]).to_batches(max_chunksize=20_000)
+        LOG.warning("--limit %d: pretrain subsample %d rows", limit, n)
+    else:
+        batches = pq.ParquetFile(path).iter_batches(batch_size=20_000, columns=["text"])
+    n_rows = 0
+    t0 = common.timer()
+    for batch in batches:
+        for text in batch.column(0).to_pylist():
+            k, base = text_identity(text or "")
+            keys |= k
+            bases.append(base[: nd.window_chars])
+            n_rows += 1
+    LOG.info("pretrain identity: %d rows, %d keys (%.1fs)", n_rows, len(keys), common.timer() - t0)
+    return keys, bases, n_rows, dev_table
+
+
+def champa_commit_sha(local_dir: Path, fname: str, repo: str) -> str | None:
+    """Commit sha recorded by hf_hub_download in <local_dir>/.cache/huggingface/download/<file>.metadata (offline), else the Hub."""
+    meta = local_dir / ".cache" / "huggingface" / "download" / f"{fname}.metadata"
+    if meta.exists():
+        first = meta.read_text(encoding="utf-8").splitlines()[0].strip()
+        if re.fullmatch(r"[0-9a-f]{40}", first):
+            return first
+    return hub_sha(repo)
+
+
+def load_champa(tw: dict, limit: int | None, rng: np.random.Generator) -> tuple[pd.DataFrame, dict]:
+    """Nazario.csv + Nigerian_Fraud.csv from the Champa et al. mirror -> raw frame (subject, body, label, source, label3).
+
+    Both files are phishing corpora by construction (all label 1 in the file, label3 = 1 by provenance). The
+    sender/receiver/date/urls columns are never used: model input is subject + body only, like the primary set.
+    Downloaded with hf_hub_download when missing; the commit sha is recorded per file.
+    """
+    c = tw["champa"]
+    local_dir = Path(c["local_dir"])
+    if not local_dir.is_absolute():
+        local_dir = common.project_root() / local_dir
+    meta: dict[str, Any] = {"repo": c["repo"], "local_dir": str(local_dir), "label3": int(c["label3"]), "files": {}}
+    frames = []
+    for fname in c["files"]:
+        path = local_dir / fname
+        if not path.exists():
+            from huggingface_hub import hf_hub_download
+
+            LOG.info("downloading %s/%s -> %s", c["repo"], fname, local_dir)
+            hf_hub_download(c["repo"], fname, repo_type="dataset", local_dir=str(local_dir))
+        sha = champa_commit_sha(local_dir, fname, c["repo"])
+        raw = pd.read_csv(path, dtype=str, keep_default_na=False)
+        missing = {"subject", "body", "label"} - set(raw.columns)
+        if missing:
+            raise ValueError(f"{path}: missing columns {sorted(missing)}")
+        labels = raw["label"].astype(np.int64).to_numpy()
+        if (labels != 1).any():
+            raise ValueError(f"{path}: expected every label == 1 (phishing corpus), found {Counter(labels.tolist())}")
+        stem = Path(fname).stem
+        df = pd.DataFrame({
+            "doc_id": [f"aura:{stem}:{i:06d}" for i in range(len(raw))],
+            "subject": raw["subject"].astype(str), "body": raw["body"].astype(str),
+            "label": np.ones(len(raw), dtype=np.int64), "source": stem,
+            "label3": np.full(len(raw), int(c["label3"]), dtype=np.int64),
+        })
+        if limit:
+            df = df.iloc[np.sort(rng.choice(len(df), min(limit, len(df)), replace=False))].reset_index(drop=True)
+        meta["files"][fname] = {"rows": int(len(raw)), "rows_used": int(len(df)), "commit_sha": sha, "path": str(path),
+                                "bytes": int(path.stat().st_size)}
+        LOG.info("champa %s @ %s: %d rows (%d used), %d null subjects", fname, sha, len(raw), len(df), int((raw["subject"] == "").sum()))
+        frames.append(df)
+    return pd.concat(frames, ignore_index=True), meta
+
+
+def english_vocab(train: pd.DataFrame, rng: np.random.Generator, sample: int = 20_000, min_count: int = 3) -> set[str]:
+    """Lower-case word types seen >= min_count times in a sample of legitimate training bodies (bigshp filter)."""
+    legit = np.flatnonzero((train["label"] == 0).to_numpy())
+    idx = rng.choice(legit, min(sample, legit.size), replace=False)
+    counts: Counter = Counter()
+    for body in train["body"].to_numpy()[idx].tolist():
+        counts.update(_WORD.findall(body.lower()))
+    return {w for w, n in counts.items() if n >= min_count}
+
+
+def load_bigshp(tw: dict, limit: int | None, rng: np.random.Generator, vocab: set[str]) -> tuple[pd.DataFrame, dict]:
+    """locuoco/...-300000 df.csv behind data.threeway.bigshp.enabled, with the research verdict's filter chain.
+
+    Research verdict (docs/research/bigshp_research.md): DO_NOT_USE. The file is a union of lossy re-packagings of
+    corpora we already hold; its phish label is source-derived (87% of resolvable phish rows are spam-folder or
+    spam-track mail) and the whole phish class went through a stopword-stripping pipeline that would act as a label
+    shortcut. This loader exists so the decision is reversible from config.yaml. Filters, in order:
+    drop the label-1 class; drop rows shorter than min_chars; label-2 rows with any non-ASCII byte or < min_english_fraction
+    of word tokens in our legitimate-mail vocabulary; label-0 rows that look like Ling-Spam list mail (first token
+    "subject", no punctuation); within-file dedup on k_full, k_pre and the first-N-token key. The cross-set screens
+    (keys and shingles against train/val/holdout/pretrain) then apply exactly as for the Champa rows; a surviving
+    row whose key matches one of our rows is dropped there, with label contradictions counted separately.
+    NOTE: no digit-folding adapter ("escapenumber") is applied to our keys, so exact-key recall on bigshp text is a
+    lower bound and the shingle screen carries the near-duplicate removal.
+    """
+    import pyarrow.csv as pacsv
+
+    b = tw["bigshp"]
+    local_dir = Path(b["local_dir"])
+    if not local_dir.is_absolute():
+        local_dir = common.project_root() / local_dir
+    path = local_dir / b["file"]
+    if not path.exists():
+        from huggingface_hub import hf_hub_download
+
+        LOG.info("downloading %s/%s -> %s", b["repo"], b["file"], local_dir)
+        hf_hub_download(b["repo"], b["file"], repo_type="dataset", local_dir=str(local_dir))
+    sha = champa_commit_sha(local_dir, b["file"], b["repo"])
+    table = pacsv.read_csv(
+        path,
+        parse_options=pacsv.ParseOptions(newlines_in_values=True),
+        read_options=pacsv.ReadOptions(block_size=64 << 20),
+        convert_options=pacsv.ConvertOptions(column_types={"label": pa.int64(), "text": pa.large_string()}),
+    )
+    n_total = table.num_rows
+    if limit:
+        idx = np.sort(rng.choice(n_total, min(limit * 4, n_total), replace=False))
+        table = table.take(pa.array(idx))
+        LOG.warning("--limit %d: bigshp subsample %d rows", limit, table.num_rows)
+    row_idx = idx if limit else np.arange(n_total)
+    label3 = np.asarray(table.column("label").to_pylist(), dtype=np.int64)
+    texts = [t or "" for t in table.column("text").to_pylist()]
+    del table
+    n = len(texts)
+    lengths = np.fromiter((len(t) for t in texts), dtype=np.int64, count=n)
+    non_ascii = np.fromiter((not t.isascii() for t in texts), dtype=bool, count=n)
+    eng = np.ones(n, dtype=np.float64)
+    ling_like = np.zeros(n, dtype=bool)
+    tok_keys: list[str] = []
+    k = int(b["near_dup_token_key_len"])
+    for i, t in enumerate(texts):
+        words = _WORD.findall(t.lower())
+        if words:
+            eng[i] = sum(1 for w in words if w in vocab) / len(words)
+        tok_keys.append(common.sha1_hex(" ".join(words[:k])))
+        if label3[i] == 0:
+            first = t.strip().split(None, 1)[0].lower() if t.strip() else ""
+            ling_like[i] = first == "subject" and _PUNCT.search(t) is None
+    keyed = pd.DataFrame({"subject": [""] * n, "body": texts})
+    add_key_columns(keyed)
+    reasons = [
+        ("bigshp_phish_class_source_derived", (label3 == 1) if b["drop_phish_class"] else np.zeros(n, dtype=bool)),
+        ("bigshp_label_outside_0_1_2", ~np.isin(label3, [0, 1, 2])),
+        (f"bigshp_shorter_than_{int(b['min_chars'])}_chars", lengths < int(b["min_chars"])),
+        ("bigshp_spam_non_ascii", (label3 == 2) & non_ascii),
+        (f"bigshp_spam_english_fraction_lt_{b['min_english_fraction']}", (label3 == 2) & (eng < float(b["min_english_fraction"]))),
+        ("bigshp_ham_ling_list_mail", (label3 == 0) & ling_like),
+        ("bigshp_within_file_k_full_dup", keyed["k_full"].duplicated(keep="first").to_numpy()),
+        ("bigshp_within_file_k_pre_dup", keyed["k_pre"].duplicated(keep="first").to_numpy()),
+        (f"bigshp_within_file_first{k}_token_dup", pd.Series(tok_keys).duplicated(keep="first").to_numpy()),
+    ]
+    reason = first_reason(n, reasons)
+    keep = reason == ""
+    df = pd.DataFrame({
+        "doc_id": [f"bigshp:{int(i):06d}" for i in row_idx[keep].tolist()],
+        "subject": "", "body": [t for t, kp in zip(texts, keep) if kp],
+        "label": (label3[keep] != 0).astype(np.int64), "source": BIGSHP_SOURCE, "label3": label3[keep],
+    })
+    meta = {
+        "repo": b["repo"], "file": b["file"], "commit_sha": sha, "path": str(path), "bytes": int(path.stat().st_size),
+        "rows_total": int(n_total), "rows_read": int(n), "rows_after_file_filters": int(len(df)),
+        "file_filter_drops": {name: int(v) for name, v in Counter(reason[~keep].tolist()).items()},
+        "filters": {key: b[key] for key in ("drop_phish_class", "min_chars", "min_english_fraction", "near_dup_token_key_len")},
+        "label_counts_read": {str(int(c)): int(v) for c, v in Counter(label3.tolist()).items()},
+    }
+    LOG.info("bigshp @ %s: %d rows read, %d survive the file filters %s", sha, n, len(df), meta["file_filter_drops"])
+    return df, meta
+
+
+def prepare_new_rows(df: pd.DataFrame) -> None:
+    """Clean exactly like the primary set: clean_text, model_input, keys/base, label3 -> label_set."""
+    df["subject"] = df["subject"].map(clean_text)
+    df["body"] = df["body"].map(clean_text)
+    df["text"] = [model_input(s, b) for s, b in zip(df["subject"].tolist(), df["body"].tolist())]
+    df["label"] = df["label"].astype(np.int64)
+    df["source"] = df["source"].astype(str)
+    df["base"] = add_key_columns(df)
+    add_label3_columns(df, df["label3"].to_numpy())
+
+
+# ----------------------------------------------------------------------------- three-way: leakage
+def recount_links_multi(parts: dict[str, list[str]], pretrain_bases: list[str], pretrain_ref: tuple[str, ...], nd: NearDupConfig) -> dict[str, int]:
+    """Independent near-dup link counts between every pair of final sets, from a fresh index.
+
+    Generalises recount_shingle_links to four parts (holdout, test3, train, val): one index over all parts
+    enumerates every cross-part link; the pretraining corpus is queried against an index of the small reference
+    parts so pretrain-pretrain links are never materialised. Pair names follow PART_ORDER (holdout_train,
+    test3_val, ...), so the binary gate's keys (holdout_train, holdout_val, train_val, holdout_pretrain,
+    val_pretrain) come out unchanged.
+    """
+    names = [p for p in PART_ORDER if p in parts]
+    bounds = np.cumsum([0] + [len(parts[n]) for n in names])
+    index = ShingleIndex([b for n in names for b in parts[n]], nd)
+    la, lb = index.links()
+    pa_ = np.searchsorted(bounds, la, side="right") - 1
+    pb_ = np.searchsorted(bounds, lb, side="right") - 1
+    pair = Counter(zip(np.minimum(pa_, pb_).tolist(), np.maximum(pa_, pb_).tolist()))
+    out = {f"{names[i]}_{names[j]}": int(pair[(i, j)]) for i in range(len(names)) for j in range(i + 1, len(names))}
+    del index, la, lb
+    ref_names = [n for n in pretrain_ref if n in parts]
+    ref = ShingleIndex([b for n in ref_names for b in parts[n]], nd)
+    _q, r = ref.query(pretrain_bases)
+    rb = np.cumsum([0] + [len(parts[n]) for n in ref_names])
+    part_of = np.searchsorted(rb, r, side="right") - 1
+    for i, n in enumerate(ref_names):
+        out[f"{n}_pretrain"] = int((part_of == i).sum())
+    LOG.info("recount shingle links (final sets): %s", out)
+    return out
+
+
+def leakage_checks3(train: pd.DataFrame, val: pd.DataFrame, holdout: pd.DataFrame, test3: pd.DataFrame,
+                    pretrain_keys: set[str], n_pretrain: int, links: dict[str, int], holdout_source: str,
+                    pos_map: dict[str, int]) -> LeakageResults:
+    """The binary gate (unchanged) plus the test_holdout3 assertions.
+
+    test_holdout3 must be text the model never saw in any form: zero exact keys and zero >=3-shingle links against
+    train, val and the MLM corpus, zero keys shared with the TREC-06 holdout (its shingle links to TREC-06 are
+    counted only: the two test sets may legitimately contain the same 1998-2008 spam templates), every row fully
+    labelled (label3 in {0,1,2}), no TREC-06 row, no doc_id in two splits, no contract violation anywhere.
+    """
+    res = leakage_checks(train, val, holdout, pretrain_keys, n_pretrain, links, holdout_source)
+    t3_keys = key_union(test3)
+    hold_keys = key_union(holdout)
+    res.add("holdout3_vs_train_key_rows", int(rows_hitting(train, t3_keys).sum()), True)
+    res.add("holdout3_vs_val_key_rows", int(rows_hitting(val, t3_keys).sum()), True)
+    res.add("holdout3_vs_pretrain_key_rows", len(pretrain_keys & t3_keys), True, "pretrain keys re-derived from stored text (text_identity)")
+    res.add("holdout3_vs_holdout_key_rows", int(rows_hitting(test3, hold_keys).sum()), True)
+    res.add("holdout3_vs_train_shingle_links", links["test3_train"], True)
+    res.add("holdout3_vs_val_shingle_links", links["test3_val"], True)
+    res.add("holdout3_vs_pretrain_shingle_links", links["test3_pretrain"], True, "pretrain bases re-derived from stored text (text_identity)")
+    res.add("holdout3_vs_holdout_shingle_links", links["holdout_test3"], False, "two test sets; shared 1998-2008 spam templates are allowed")
+    res.add("holdout3_rows_without_full_label3", int((~test3["label3"].isin([0, 1, 2])).sum()), True)
+    res.add("holdout3_rows_from_holdout_source", int((test3["source"] == holdout_source).sum()), True)
+    ids = [set(df["doc_id"].tolist()) for df in (train, val, holdout, test3)]
+    res.add("doc_id_shared_between_splits", sum(len(ids[i] & ids[j]) for i in range(4) for j in range(i + 1, 4)), True)
+    viol = {name: contract_violations(df, pos_map) for name, df in zip(THREEWAY_SPLITS, (train, val, holdout, test3))}
+    res.add("label3_contract_violations", sum(sum(v.values()) for v in viol.values()), True, str({k: v for k, v in viol.items() if sum(v.values())}))
+    res.add("holdout3_rows", len(test3), False)
+    return res
+
+
+# ----------------------------------------------------------------------------- three-way: build
+def cmd_build3(args: argparse.Namespace, cfg: dict) -> None:
+    """Derive the three-way files from the binary splits + the Champa phishing corpora (+ bigshp if enabled).
+
+    Binary split membership is preserved except for the rows moved from finetune_val into test_holdout3; every
+    other existing row only gains label3/label_set. New rows are cleaned, keyed, deduplicated within themselves
+    and against train/val/holdout/pretrain (exact keys), screened for >=3-shingle links against the TREC-06
+    holdout (dropped), pinned to train/val when they near-duplicate an existing train/pretrain/val row (so no
+    template straddles a split), and the free clusters are split test3 / val / train by grouped_stratified_split.
+    """
+    t_start = common.timer()
+    seed = int(cfg["seed"])
+    common.set_seed(seed)
+    rng = np.random.default_rng(seed + 3)       # independent stream from `build` so the binary split is never re-drawn
+    tw = threeway_config(cfg)
+    if args.bigshp:
+        tw["bigshp"]["enabled"] = args.bigshp == "on"
+    pos_map = {str(k): int(v) for k, v in tw["positive_label3_by_source"].items()}
+    nd = NearDupConfig.from_config(cfg, cap=args.shingle_cap or DIALS.shingle_cap)
+    holdout_source = cfg["data"]["holdout_test_source"]
+    min_alnum = int(cfg["data"]["min_alnum_chars"])
+    processed_dir = Path(cfg["paths"]["data_processed"])
+    dev = bool(args.limit)
+    out_dir = Path(args.out_dir) if args.out_dir else (processed_dir / f"dev_threeway_limit{args.limit}" if dev else processed_dir)
+    out_dir.mkdir(parents=True, exist_ok=True)
+    LOG.info("build3: limit=%s seed=%d out_dir=%s bigshp.enabled=%s holdout=%s", args.limit, seed, out_dir, tw["bigshp"]["enabled"], holdout_source)
+
+    # 1. binary inputs (snapshot), keyed; label3 from provenance
+    base_dir, base_origin = binary_base_dir(processed_dir, dev)
+    tables: dict[str, pa.Table] = {}
+    frames: dict[str, pd.DataFrame] = {}
+    for name in ("finetune_train", "finetune_val", "test_holdout"):
+        tables[name], frames[name] = read_binary_split(base_dir / f"{name}.parquet", args.limit, rng)
+    train, val, holdout = frames["finetune_train"], frames["finetune_val"], frames["test_holdout"]
+    for df in (train, val, holdout):
+        add_label3_columns(df, label3_from_provenance(df["source"].to_numpy(), df["label"].to_numpy(), pos_map))
+    base_counts = {n: nested_counts3(frames[n]) for n in frames}
+    pretrain_keys, pretrain_bases, n_pretrain, pretrain_dev_table = pretrain_identity(processed_dir / "pretrain.parquet", nd, args.limit, rng)
+    LOG.info("binary base from %s: train %d val %d holdout %d pretrain %d (%.1fs)", base_origin, len(train), len(val), len(holdout), n_pretrain, common.timer() - t_start)
+
+    # 2. new sources, cleaned exactly like the primary set
+    new, champa_meta = load_champa(tw, args.limit, rng)
+    bigshp_meta: dict[str, Any] = {"enabled": bool(tw["bigshp"]["enabled"]), "verdict": tw["bigshp"]["verdict"],
+                                   "research": "docs/research/bigshp_research.md"}
+    if tw["bigshp"]["enabled"]:
+        bdf, bmeta = load_bigshp(tw, args.limit, rng, english_vocab(train, rng))
+        bigshp_meta.update(bmeta)
+        new = pd.concat([new, bdf], ignore_index=True)
+    prepare_new_rows(new)
+    loaded_by_source = {str(k): int(v) for k, v in new["source"].value_counts().items()}
+    LOG.info("new rows: %s", loaded_by_source)
+
+    # 3. exact screens: min-alnum, mbox artefacts, within-new dups, keys against holdout / train / val / pretrain
+    hold_keys = frozenset(key_union(holdout))
+    train_keys = frozenset(key_union(train))
+    val_keys = frozenset(key_union(val))
+    hit_hold, hit_train, hit_val, hit_pre = (rows_hitting(new, ks) for ks in (hold_keys, train_keys, val_keys, pretrain_keys))
+    reasons = [
+        (f"lt_{min_alnum}_alnum", (new["n_alnum"] < min_alnum).to_numpy()),
+        ("mbox_internal_data", (new["subject"] == MBOX_INTERNAL_SUBJECT).to_numpy()),
+        ("within_new_k_full_dup", new["k_full"].duplicated(keep="first").to_numpy()),
+        ("key_match_test_holdout", hit_hold),
+        ("key_match_finetune_train", hit_train),
+        ("key_match_finetune_val", hit_val),
+        ("key_match_pretrain", hit_pre),
+    ]
+    reason = first_reason(len(new), reasons)
+    contradictions = 0
+    if tw["bigshp"]["enabled"]:
+        # a bigshp row that exact-matches one of our rows is dropped anyway; count how often its label would have
+        # disagreed with our provenance label (the research verdict's "label contradicts provenance" filter)
+        own_label: dict[str, int] = {}
+        for df in (train, val, holdout):
+            for col in ("k_full", "k_pre", "k_sb", "k_sb_full"):
+                own_label.update(zip(df[col].tolist(), df["label3"].tolist()))
+        for i in np.flatnonzero((new["source"] == BIGSHP_SOURCE).to_numpy() & (hit_train | hit_val | hit_hold)):
+            match = [own_label[k] for k in (new.at[i, "k_full"], new.at[i, "k_pre"], new.at[i, "k_sb"], new.at[i, "k_sb_full"]) if k in own_label]
+            if match and any(m != int(new.at[i, "label3"]) and m != -1 for m in match):
+                contradictions += 1
+    survivors = new.loc[reason == ""].reset_index(drop=True)
+    LOG.info("exact screens dropped %d of %d new rows: %s", int((reason != "").sum()), len(new), by_source_reason(new["source"].to_numpy(), reason))
+
+    # 4. one shingle index over (new survivors + val): holdout -> drop, train/pretrain -> pin train, val -> pin val
+    n_new = len(survivors)
+    ref = ShingleIndex(survivors["base"].tolist() + val["base"].tolist(), nd)
+    _q, r_hold = ref.query(holdout["base"].tolist())
+    new_hold_linked = np.zeros(n_new, dtype=bool)
+    new_hold_linked[r_hold[r_hold < n_new]] = True
+    val_hold_linked = int(np.unique(r_hold[r_hold >= n_new]).size)
+    _q, r_tr = ref.query(train["base"].tolist())
+    _q, r_pre = ref.query(pretrain_bases)
+    new_train_linked = np.zeros(n_new, dtype=bool)
+    new_train_linked[r_tr[r_tr < n_new]] = True
+    new_pre_linked = np.zeros(n_new, dtype=bool)
+    new_pre_linked[r_pre[r_pre < n_new]] = True
+    val_train_linked = np.zeros(len(val), dtype=bool)
+    val_train_linked[r_tr[r_tr >= n_new] - n_new] = True
+    val_pre_linked = np.zeros(len(val), dtype=bool)
+    val_pre_linked[r_pre[r_pre >= n_new] - n_new] = True
+    la, lb = ref.links()
+    a_new, b_new = la < n_new, lb < n_new
+    new_new = a_new & b_new
+    new_val = a_new ^ b_new
+    new_side = np.where(a_new[new_val], la[new_val], lb[new_val])
+    val_side = np.where(a_new[new_val], lb[new_val], la[new_val]) - n_new
+    new_val_linked = np.zeros(n_new, dtype=bool)
+    new_val_linked[new_side] = True
+    val_new_linked = np.zeros(len(val), dtype=bool)
+    val_new_linked[val_side] = True
+    del ref
+
+    # 5. drop holdout-linked new rows; cluster the rest (shingle links + shared k_pre / k_sb); pin clusters
+    reason_s = first_reason(n_new, [("shingle_link_test_holdout", new_hold_linked)])
+    keep_s = reason_s == ""
+    edges = [(int(a), int(b)) for a, b in zip(la[new_new].tolist(), lb[new_new].tolist()) if keep_s[a] and keep_s[b]]
+    edges += [(a, b) for a, b in same_key_pairs_multi([survivors["k_pre"].tolist(), survivors["k_sb"].tolist()]) if keep_s[a] and keep_s[b]]
+    cl = union_find(n_new, edges)
+    # A cluster that near-duplicates rows on BOTH sides of the binary train/val split (or the MLM corpus and val)
+    # is a template family the primary set already holds on both sides; placing it anywhere would create
+    # train/val near-dup links the binary build had ruled out by construction, so it is dropped instead.
+    train_side_cl = set(cl[(new_train_linked | new_pre_linked) & keep_s].tolist())
+    val_side_cl = set(cl[new_val_linked & keep_s].tolist())
+    both_cl = train_side_cl & val_side_cl if tw["drop_new_clusters_linked_to_both_train_and_val"] else set()
+    both_rows = keep_s & np.isin(cl, list(both_cl))
+    reason_s[both_rows] = "near_dup_of_both_train_and_val"
+    keep_s = reason_s == ""
+    pin_train_cl = train_side_cl - both_cl
+    pin_val_cl = val_side_cl - both_cl
+    hold_touched_cl = set(cl[new_hold_linked].tolist())
+    dest = np.full(n_new, "", dtype=object)
+    dest[keep_s & np.isin(cl, list(pin_train_cl))] = "train"
+    dest[keep_s & np.isin(cl, list(pin_val_cl))] = "val"
+    free = keep_s & (dest == "")
+    # free clusters: ~15% to test3 (cluster-grouped, stratified by source x label3), ~10% of the rest to val
+    tmp = pd.DataFrame({"source": survivors["source"], "label": survivors["label3"], "cluster_id": cl})
+    is_test, split_test_info = grouped_stratified_split(tmp.loc[free].reset_index(drop=True), float(tw["new_source_test_fraction"]), rng)
+    free_idx = np.flatnonzero(free)
+    dest[free_idx[is_test]] = "test3"
+    rest_idx = free_idx[~is_test]
+    is_val_new, split_val_info = grouped_stratified_split(tmp.iloc[rest_idx].reset_index(drop=True), float(tw["new_source_val_fraction"]), rng)
+    dest[rest_idx[is_val_new]] = "val"
+    dest[rest_idx[~is_val_new]] = "train"
+    assert ((dest != "") == keep_s).all()
+    cluster_offset = int(max(train["cluster_id"].max(), val["cluster_id"].max(), holdout["cluster_id"].max())) + 1
+    survivors["cluster_id"] = (cl + cluster_offset).astype(np.int64)
+    survivors["dest"] = dest
+    new_pins = {
+        "clusters_total": int(np.unique(cl[keep_s]).size), "clusters_pinned_train_by_train_or_pretrain_link": len(pin_train_cl),
+        "clusters_pinned_val_by_val_link": len(pin_val_cl), "clusters_dropped_linked_to_both_train_and_val": len(both_cl),
+        "rows_dropped_linked_to_both_train_and_val": int(both_rows.sum()), "clusters_with_a_holdout_linked_member": len(hold_touched_cl),
+        "rows_linked_to_train": int((new_train_linked & keep_s).sum()), "rows_linked_to_pretrain": int((new_pre_linked & keep_s).sum()),
+        "rows_linked_to_val": int((new_val_linked & keep_s).sum()), "rows_dropped_holdout_shingle_link": int(new_hold_linked.sum()),
+        "largest_cluster": int(np.bincount(cl[keep_s]).max()) if keep_s.any() else 0,
+    }
+    LOG.info("new rows: pins %s; destinations %s", new_pins, Counter(dest[keep_s].tolist()))
+    all_reason = reason.copy()
+    all_reason[np.flatnonzero(reason == "")] = reason_s
+    drops_by_source_reason = by_source_reason(new["source"].to_numpy(), all_reason)
+    new_kept = survivors.loc[keep_s].reset_index(drop=True)
+    dest_by_source = {s: {d: int(n) for d, n in grp["dest"].value_counts().items()} for s, grp in new_kept.groupby("source")}
+
+    # 6. test_holdout3 part (b): clean-provenance legit / spam rows of the CURRENT val, whole clusters only
+    h3 = tw["holdout3_from_val"]
+    l3 = val["label3"].to_numpy()
+    src = val["source"].to_numpy()
+    cand = ((l3 == 0) & np.isin(src, list(h3["legit_sources"]))) | ((l3 == 2) & np.isin(src, list(h3["spam_sources"])))
+    val_pre_key = rows_hitting(val, pretrain_keys)
+    val_hold_key = rows_hitting(val, hold_keys)
+    train_clusters = set(train["cluster_id"].tolist())
+    in_train_cl = val["cluster_id"].isin(train_clusters).to_numpy()
+    excl = first_reason(len(val), [
+        ("not_a_candidate_source_or_label", ~cand),
+        ("key_match_pretrain", val_pre_key), ("shingle_link_pretrain", val_pre_linked),
+        ("key_or_shingle_link_test_holdout", val_hold_key | (np.isin(np.arange(len(val)), r_hold[r_hold >= n_new] - n_new))),
+        ("shingle_link_train", val_train_linked), ("cluster_also_in_train", in_train_cl),
+        ("shingle_link_to_a_new_row", val_new_linked),
+    ])
+    row_ok = excl == ""
+    cluster_ok = pd.Series(row_ok).groupby(val["cluster_id"].to_numpy()).transform("all").to_numpy()
+    excl[row_ok & ~cluster_ok] = "cluster_has_an_ineligible_member"
+    eligible = excl == ""
+    moved = np.zeros(len(val), dtype=bool)
+    h3_split_info = {}
+    for cls, target_rows in ((0, int(h3["legit_rows"])), (2, int(h3["spam_rows"]))):
+        pool_mask = eligible & (l3 == cls)
+        pool_idx = np.flatnonzero(pool_mask)
+        if pool_idx.size == 0:
+            LOG.warning("holdout3 from val: no eligible rows for label3=%d", cls)
+            continue
+        frac = min(1.0, target_rows / pool_idx.size)
+        tmp = pd.DataFrame({"source": src[pool_idx], "label": l3[pool_idx], "cluster_id": val["cluster_id"].to_numpy()[pool_idx]})
+        sel, info = grouped_stratified_split(tmp, frac, rng)
+        moved[pool_idx[sel]] = True
+        h3_split_info[LABEL3_NAMES[cls]] = {"target_rows": target_rows, "eligible_rows": int(pool_idx.size), "fraction": round(frac, 4), "moved_rows": int(sel.sum()), "split": info}
+    excl_counts = by_source_reason(src, np.where(cand, excl, ""))
+    LOG.info("holdout3 from val: candidates %d, eligible %d, moved %d; exclusions %s", int(cand.sum()), int(eligible.sum()), int(moved.sum()), excl_counts)
+
+    # 7. final frames (keyed) for the independent recount + gate
+    cols = THREEWAY_SCHEMA[:1] + ["subject", "body", "label", "source", "cluster_id", "label3", "label_set", "base", "k_full", "k_pre", "k_sb", "k_sb_full", "n_alnum"]
+    new_train = new_kept.loc[new_kept["dest"] == "train"]
+    new_val = new_kept.loc[new_kept["dest"] == "val"]
+    new_test = new_kept.loc[new_kept["dest"] == "test3"]
+    train_f = pd.concat([train[cols], new_train[cols]], ignore_index=True)
+    val_f = pd.concat([val.loc[~moved, cols], new_val[cols]], ignore_index=True)
+    test3 = pd.concat([val.loc[moved, cols], new_test[cols]], ignore_index=True)
+    links = recount_links_multi(
+        {"train": train_f["base"].tolist(), "val": val_f["base"].tolist(), "holdout": holdout["base"].tolist(), "test3": test3["base"].tolist()},
+        pretrain_bases, ("holdout", "val", "test3"), nd,
+    )
+    del pretrain_bases
+    leak = leakage_checks3(train_f, val_f, holdout, test3, pretrain_keys, n_pretrain, links, holdout_source, pos_map)
+    leak.raise_if_failed()
+
+    # 8. write (atomic): existing rows keep their text byte-for-byte (arrow tables), new rows appended
+    def out_table(base: pa.Table, keep: np.ndarray | None, frame: pd.DataFrame, extra: pd.DataFrame | None, tag: str) -> pa.Table:
+        t = base.select(FINETUNE_SCHEMA)
+        if keep is not None:
+            t = t.filter(pa.array(keep))
+        if t.num_rows != len(frame) or (t.num_rows and t.column("doc_id")[0].as_py() != frame["doc_id"].iloc[0]):
+            raise AssertionError(f"{tag}: arrow table / frame misaligned")
+        t = t.append_column("label3", pa.array(frame["label3"].to_numpy(), pa.int64()))
+        t = t.append_column("label_set", pa.array(frame["label_set"].tolist(), pa.large_string()))
+        if extra is not None and len(extra):
+            ex = pa.Table.from_pandas(extra[THREEWAY_SCHEMA].reset_index(drop=True), preserve_index=False).cast(t.schema)
+            t = pa.concat_tables([t, ex]).combine_chunks()
+        meta = {b"threeway": json.dumps({"label3": SET_OF_LABEL3, "display_names": LABEL3_NAMES, "binary_rollup": UNWANTED_NAME, "built_by": "data_prep.py build3"}).encode()}
+        return t.replace_schema_metadata(meta)
+
+    out_tables = {
+        "finetune_train": out_table(tables["finetune_train"], None, train, new_train, "train"),
+        "finetune_val": out_table(tables["finetune_val"], ~moved, val.loc[~moved], new_val, "val"),
+        "test_holdout": out_table(tables["test_holdout"], None, holdout, None, "holdout"),
+        "test_holdout3": out_table(tables["finetune_val"], moved, val.loc[moved], new_test, "test3"),
+    }
+    files: dict[str, str] = {}
+    for name, t in out_tables.items():
+        path = out_dir / f"{name}.parquet"
+        write_table_atomic(t, path)
+        files[name] = str(path)
+        LOG.info("wrote %s (%d rows, %s)", path, t.num_rows, Counter(t.column("label3").to_pylist()))
+    if pretrain_dev_table is not None:
+        write_table_atomic(pretrain_dev_table, out_dir / "pretrain.parquet")
+        files["pretrain"] = str(out_dir / "pretrain.parquet")
+    del tables, out_tables
+
+    # 9. manifest section + report section
+    finals = {"finetune_train": train_f, "finetune_val": val_f, "test_holdout": holdout, "test_holdout3": test3}
+    elapsed = round(common.timer() - t_start, 1)
+    section = {
+        "timestamp": datetime.now(timezone.utc).isoformat(timespec="seconds"),
+        "limit": args.limit, "dev_subsample": dev, "seed": seed, "rng_stream": "default_rng(seed + 3)",
+        "elapsed_seconds": elapsed, "peak_rss_gb": peak_rss_gb(),
+        "contract": {"label3": {str(k): v for k, v in SET_OF_LABEL3.items()}, "display_names": {str(k): v for k, v in LABEL3_NAMES.items()},
+                     "binary_rollup_name": UNWANTED_NAME, "positive_label3_by_source": pos_map,
+                     "binary_label": "unchanged: 1 for phish / spam / unwanted, 0 for legit"},
+        "binary_base": {"from": base_origin, "dir": str(base_dir), "counts_by_source_label3": base_counts,
+                        "rows": {n: int(len(frames[n])) for n in frames}},
+        "sources": {"champa": champa_meta, "bigshp": bigshp_meta},
+        "new_rows": {"loaded_by_source": loaded_by_source, "drops_by_source_reason": drops_by_source_reason,
+                     "bigshp_label_contradictions_among_key_matches": int(contradictions),
+                     "kept_by_source": {str(k): int(v) for k, v in new_kept["source"].value_counts().items()},
+                     "destination_by_source": dest_by_source, "pins": new_pins,
+                     "test_fraction": tw["new_source_test_fraction"], "val_fraction_of_remainder": tw["new_source_val_fraction"],
+                     "split_test3": split_test_info, "split_val": split_val_info, "cluster_id_offset": cluster_offset},
+        "holdout3_from_val": {"legit_sources": h3["legit_sources"], "spam_sources": h3["spam_sources"],
+                              "candidates_by_source_label3": nested_counts3(val.loc[cand]),
+                              "excluded_candidates_by_source_reason": excl_counts,
+                              "eligible_by_source_label3": nested_counts3(val.loc[eligible]),
+                              "moved_by_source_label3": nested_counts3(val.loc[moved]), "per_class": h3_split_info,
+                              "val_rows_linked_to_holdout_info": val_hold_linked},
+        "counts": {n: nested_counts3(df) for n, df in finals.items()},
+        "totals": {n: split_totals3(df) for n, df in finals.items()},
+        "shingle_links": links, "near_dup": nd.__dict__, "leakage": leak.checks, "files": files,
+        "config_snapshot": tw,
+    }
+    manifest_src = processed_dir / "splits_manifest.json"
+    manifest = common.read_json(manifest_src) if manifest_src.exists() else {}
+    manifest["threeway"] = section
+    tmp_path = out_dir / "splits_manifest.json.tmp"
+    common.write_json(manifest, tmp_path)
+    os.replace(tmp_path, out_dir / "splits_manifest.json")
+    report_src = processed_dir / "data_report.md"
+    report = report_src.read_text(encoding="utf-8") if report_src.exists() else "# Data report\n"
+    marker = "\n## Three-way labels"
+    if marker in report:
+        report = report[: report.index(marker)].rstrip("\n") + "\n"
+    write_text_atomic(report.rstrip("\n") + "\n" + render_threeway_report(section), out_dir / "data_report.md")
+    LOG.info("wrote %s and %s in %.1fs (peak RSS %.2f GB)", out_dir / "splits_manifest.json", out_dir / "data_report.md", elapsed, section["peak_rss_gb"])
+    print_summary3(section)
+
+
+def render_threeway_report(s: dict) -> str:
+    lines = [f"\n## Three-way labels — build3 {s['timestamp']}", ""]
+    if s["dev_subsample"]:
+        lines += [f"**DEV SUBSAMPLE (--limit {s['limit']}): counts below are NOT the real build.**", ""]
+    c = s["contract"]
+    lines += [
+        f"label3 {c['label3']}; display names {c['display_names']}; binary roll-up \"{c['binary_rollup_name']}\" = P(phishing)+P(spam). "
+        f"Positive rows by provenance: {c['positive_label3_by_source']}. {c['binary_label']}. "
+        f"Elapsed {s['elapsed_seconds']}s; peak RSS {s['peak_rss_gb']} GB; binary base from {s['binary_base']['from']} (`{s['binary_base']['dir']}`).",
+        "",
+        "### New sources",
+    ]
+    ch = s["sources"]["champa"]
+    for fname, f in ch["files"].items():
+        lines.append(f"- `{ch['repo']}` / `{fname}` @ `{f['commit_sha']}`: {f['rows']:,} rows ({f['rows_used']:,} used), label3 {ch['label3']} (phishing by provenance)")
+    bg = s["sources"]["bigshp"]
+    if bg["enabled"]:
+        lines.append(f"- bigshp `{bg['repo']}` @ `{bg.get('commit_sha')}` ENABLED despite verdict {bg['verdict']}: {bg['rows_read']:,} rows read, "
+                     f"{bg['rows_after_file_filters']:,} after the file filters {bg['file_filter_drops']}; filters {bg['filters']}")
+    else:
+        lines.append(f"- bigshp (`{bg.get('repo', 'locuoco/the-biggest-spam-ham-phish-email-dataset-300000')}`): research verdict **{bg['verdict']}**, not ingested (`{bg['research']}`)")
+    nr = s["new_rows"]
+    rows = []
+    for src in sorted(nr["loaded_by_source"]):
+        d = nr["drops_by_source_reason"].get(src, {})
+        dest = nr["destination_by_source"].get(src, {})
+        rows.append([src, nr["loaded_by_source"][src], sum(d.values()), nr["kept_by_source"].get(src, 0), dest.get("train", 0), dest.get("val", 0), dest.get("test3", 0)])
+    lines += ["", md_table(["source", "loaded", "dropped", "kept", "-> train", "-> val", "-> test_holdout3"], rows),
+              "", f"- drops by reason: {nr['drops_by_source_reason']}", f"- near-dup pins: {nr['pins']}",
+              f"- bigshp label contradictions among exact matches: {nr['bigshp_label_contradictions_among_key_matches']}",
+              "", "### test_holdout3 rows taken from finetune_val",
+              f"- legitimate from {s['holdout3_from_val']['legit_sources']}, spam from {s['holdout3_from_val']['spam_sources']}; whole near-dup clusters only; "
+              f"excluded candidates by reason: {s['holdout3_from_val']['excluded_candidates_by_source_reason']}",
+              f"- per class: { {k: {kk: vv for kk, vv in v.items() if kk != 'split'} for k, v in s['holdout3_from_val']['per_class'].items()} }",
+              source_label3_table(s["holdout3_from_val"]["moved_by_source_label3"]), "", "### Splits by label3"]
+    for name in THREEWAY_SPLITS:
+        t = s["totals"][name]
+        lines += [f"#### {name}: {t['rows']:,} rows, by label3 {t['by_label3']}, binary positives {t['binary_positives']:,} ({t['binary_base_rate']})",
+                  source_label3_table(s["counts"][name]), ""]
+    lines += ["### Leakage assertions (binary gate + three-way)",
+              md_table(["check", "count", "asserted zero", "pass"], [[k, v["count"], v["asserted"], v["pass"]] for k, v in s["leakage"].items()]),
+              "", "### Files"] + [f"- `{k}`: `{v}`" for k, v in s["files"].items()]
+    return "\n".join(lines) + "\n"
+
+
+def print_summary3(s: dict) -> None:
+    print("\n=== data_prep build3 summary (three-way) ===")
+    if s["dev_subsample"]:
+        print(f"*** DEV SUBSAMPLE --limit {s['limit']} — not a real build ***")
+    for name in THREEWAY_SPLITS:
+        t = s["totals"][name]
+        print(f"{name:>16}: rows {t['rows']:>7,}  by label3 {t['by_label3']}  binary positives {t['binary_positives']:,}")
+        for src, cnt in sorted(s["counts"][name].items()):
+            print(f"{'':>18}{src:<16} {cnt}")
+    print("leakage:")
+    for k, v in s["leakage"].items():
+        print(f"  {k:<44} {v['count']:>8}  {'ASSERT' if v['asserted'] else 'info  '}  {'ok' if v['pass'] else 'FAIL'}")
+
+
+# ----------------------------------------------------------------------------- three-way: check
+def cmd_check3(args: argparse.Namespace, cfg: dict) -> None:
+    """Recompute every three-way assertion from the written files (no reuse of build3 bookkeeping)."""
+    processed_dir = Path(cfg["paths"]["data_processed"])
+    out_dir = Path(args.out_dir) if args.out_dir else processed_dir
+    manifest = common.read_json(out_dir / "splits_manifest.json")
+    if "threeway" not in manifest:
+        raise AssertionError(f"{out_dir / 'splits_manifest.json'} has no 'threeway' section: run build3 first")
+    section = manifest["threeway"]
+    tw = threeway_config(cfg)
+    pos_map = {str(k): int(v) for k, v in tw["positive_label3_by_source"].items()}
+    nd = NearDupConfig.from_config(cfg, cap=int(section["near_dup"]["cap"]))
+    holdout_source = cfg["data"]["holdout_test_source"]
+    frames: dict[str, pd.DataFrame] = {}
+    for name in THREEWAY_SPLITS:
+        df = pd.read_parquet(out_dir / f"{name}.parquet")
+        if list(df.columns) != THREEWAY_SCHEMA:
+            raise AssertionError(f"{name}: expected columns {THREEWAY_SCHEMA}, got {list(df.columns)}")
+        df["subject"] = df["subject"].fillna("")
+        df["body"] = df["body"].fillna("")
+        df["label"] = df["label"].astype(np.int64)
+        df["label3"] = df["label3"].astype(np.int64)
+        df["source"] = df["source"].astype(str)
+        df["base"] = add_key_columns(df)
+        frames[name] = df
+        LOG.info("read %s: %d rows, label3 %s", name, len(df), dict(Counter(df["label3"].tolist())))
+    pre_path = out_dir / "pretrain.parquet" if (out_dir / "pretrain.parquet").exists() else processed_dir / "pretrain.parquet"
+    pretrain_keys, pretrain_bases, n_pretrain, _ = pretrain_identity(pre_path, nd, None, np.random.default_rng(0))
+    train, val, hold, test3 = (frames[n] for n in THREEWAY_SPLITS)
+    links = recount_links_multi(
+        {"train": train["base"].tolist(), "val": val["base"].tolist(), "holdout": hold["base"].tolist(), "test3": test3["base"].tolist()},
+        pretrain_bases, ("holdout", "val", "test3"), nd,
+    )
+    del pretrain_bases
+    leak = leakage_checks3(train, val, hold, test3, pretrain_keys, n_pretrain, links, holdout_source, pos_map)
+
+    # the TREC-06 holdout must be the binary file with two columns added: same rows, same order, same labels
+    snap = processed_dir / BINARY_SNAPSHOT_DIR / "test_holdout.parquet"
+    if snap.exists() and not section["dev_subsample"]:
+        ref = pq.read_table(snap, columns=["doc_id", "label"])
+        same = ref.column("doc_id").to_pylist() == hold["doc_id"].tolist() and ref.column("label").to_pylist() == hold["label"].tolist()
+        leak.add("test_holdout_rows_changed_vs_binary_snapshot", 0 if same else 1, True, str(snap))
+        ref_tr = pq.read_table(processed_dir / BINARY_SNAPSHOT_DIR / "finetune_train.parquet", columns=["doc_id"]).column("doc_id").to_pylist()
+        ref_va = pq.read_table(processed_dir / BINARY_SNAPSHOT_DIR / "finetune_val.parquet", columns=["doc_id"]).column("doc_id").to_pylist()
+        binary_ids = set(ref_tr) | set(ref_va)
+        now_ids = set(train["doc_id"]) | set(val["doc_id"]) | set(test3["doc_id"])
+        leak.add("binary_train_val_rows_missing_from_train_val_holdout3", len(binary_ids - now_ids), True, "every binary train/val row must still exist in train, val or test_holdout3")
+        leak.add("binary_train_rows_not_in_train", len(set(ref_tr) - set(train["doc_id"])), True, "train membership is only ever extended")
+    for name, df in frames.items():
+        want = section["totals"][name]["rows"]
+        if want != len(df):
+            leak.add(f"{name}_rows_vs_manifest", abs(want - len(df)), True, f"manifest {want} vs file {len(df)}")
+    syn = processed_dir / "synthetic.parquet"
+    if syn.exists():
+        st = pq.read_table(syn)
+        if {"label3", "label_set", "label"} <= set(st.schema.names):
+            sdf = st.select(["label", "label3", "label_set"]).to_pandas()
+            sdf["source"] = "synthetic"
+            v = contract_violations(sdf, {"synthetic": 1})
+            leak.add("synthetic_label3_contract_violations", sum(v.values()), True, str(v))
+        else:
+            leak.add("synthetic_missing_label3_columns", 1, False, "synthetic.parquet predates the three-way contract; finetune --synthetic on will refuse it")
+    print_summary3({**section, "leakage": leak.checks})
+    print("\nmanifest threeway timestamp:", section["timestamp"], "| limit:", section["limit"], "| bigshp enabled:", section["sources"]["bigshp"]["enabled"])
+    print("build-time results:", {k: v["count"] for k, v in section["leakage"].items() if v["asserted"]})
+    leak.raise_if_failed()
+    print("ALL THREE-WAY LEAKAGE ASSERTIONS PASS")
+
+
+# ----------------------------------------------------------------------------- main
+def main(argv: list[str] | None = None) -> None:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--config", default="config.yaml")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    sub.add_parser("inspect", help="features + 2 rows of all four datasets (streaming)")
+    b = sub.add_parser("build", help="build processed parquet files + manifest + report")
+    b.add_argument("--limit", type=int, default=None, help="DEV ONLY: N rows per labelled source, 4N corbt rows")
+    b.add_argument("--spam-policy", choices=["positive_class", "map_to_zero", "exclude"], default=None)
+    b.add_argument("--no-strip-quotes", action="store_true", help="keep quoted/forwarded history in corbt bodies")
+    b.add_argument("--shingle-cap", type=int, default=None, help="ignore shingles shared by more rows than this (default: config data.dedup.near_dup_shingle_cap, else 2000)")
+    sub.add_parser("check", help="re-run leakage assertions on the written files")
+    b3 = sub.add_parser("build3", help="three-way labels: label3/label_set + Champa phishing corpora + test_holdout3 (reads the processed binary files)")
+    b3.add_argument("--limit", type=int, default=None, help="DEV ONLY: N rows per source of every split, N per new file, 4N pretrain rows; writes to data/processed/dev_threeway_limitN/")
+    b3.add_argument("--out-dir", default=None, help="output directory (default: data.processed, or the dev dir with --limit)")
+    b3.add_argument("--bigshp", choices=["on", "off"], default=None, help="override data.threeway.bigshp.enabled (research verdict: DO_NOT_USE)")
+    b3.add_argument("--shingle-cap", type=int, default=None, help="as for build")
+    c3 = sub.add_parser("check3", help="recompute the binary + three-way leakage assertions on the written files")
+    c3.add_argument("--out-dir", default=None, help="directory written by build3 (default: data.processed)")
+    args = ap.parse_args(argv)
+
+    cfg = common.load_config(args.config)
+    global LOG, DIALS
+    LOG = common.setup_logging("data_prep", cfg["paths"]["logs"])
+    DIALS = Dials.from_config(cfg)
+    LOG.info("dials: %s", DIALS)
+    {"inspect": cmd_inspect, "build": cmd_build, "check": cmd_check, "build3": cmd_build3, "check3": cmd_check3}[args.cmd](args, cfg)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/presentation/src/evaluate.py b/presentation/src/evaluate.py
new file mode 100644
index 0000000000000000000000000000000000000000..22829d92ff4b42096185c3527dc25ae36feefbd7
--- /dev/null
+++ b/presentation/src/evaluate.py
@@ -0,0 +1,1577 @@
+"""Evaluate a fine-tuned classifier on the unseen test sets (or the val split).
+
+This is the ONLY script allowed to read data/processed/test_holdout.parquet (TREC-06) and
+data/processed/test_holdout3.parquet (the three-way holdout). Everything upstream
+(data_prep, tokenizer, pretrain, finetune) is forbidden from touching them, so the numbers
+written here are the project's honest generalisation estimate.
+
+Subcommand ``run --checkpoint <finetune best dir> [--split test|test3|val|test,test3]``:
+
+1. load ``EncoderForClassification`` from ``config.json`` + ``model.safetensors`` (strict;
+   ``num_labels`` comes from ``config.json["model"]``), batched inference under bf16
+   autocast with length-bucketed dynamic padding and bool key-padding masks -> the full
+   softmax ``P(class)`` for every row.
+2. **Binary roll-up, identical to the binary-era report.** ``P(unwanted) = P(phishing) +
+   P(spam)`` (for a 2-class checkpoint this is exactly ``P(class 1)``), so accuracy,
+   precision, recall, F1, specificity, ROC-AUC, PR-AUC, recall at 1% / 0.1% FPR and the
+   best-F1 threshold stay comparable with every earlier run. **Recall on the unwanted
+   class is reported first**: in a defensive triage tool a missed phish (false negative)
+   is the costly error, a false positive costs a human a glance. Plus: a credential-lure
+   slice (``CREDENTIAL_LURE_PHRASES``), a per-source table, and the token-length confound
+   check that RUN_LOG entry 2 asks for.
+3. **Three-way metrics** (RUN_LOG entry 6) on the rows whose ``label3`` is in {0, 1, 2}:
+   3x3 confusion matrix (plot + table), per-class precision / recall / F1, macro-F1, and a
+   phishing-vs-spam confusion restricted to the true unwanted rows. Rows labelled
+   ``-1`` / ``"unwanted"`` (TREC-05 / TREC-07 / CEAS-08 positives, whose corpora never
+   separated spam from phishing) only enter the binary roll-up. The three-way decision
+   rule is **roll-up first**: a row is unwanted iff ``P(unwanted) >= threshold``, and then
+   the kind is ``argmax(P(phishing), P(spam))``, so the 3x3 matrix collapses exactly onto
+   the binary confusion matrix above; plain argmax-of-3 is reported next to it. On the
+   TREC-06 split there is no phishing/spam ground truth, so only an *informational*
+   breakdown of how the flagged rows split into predicted phishing vs spam is written,
+   and it says so. A 2-class checkpoint skips every three-way section with a log line.
+4. single-email latency like the instructor's step 5: tokenize + forward + softmax at
+   batch 1 timed with ``timeit.default_timer``, GPU bf16 and CPU fp32 (16 threads),
+   printed next to the 74 ms benchmark -> ``latency.json`` (measured once per invocation).
+5. plots (confusion matrices, ROC, PR), ``error_analysis.md`` (top false negatives /
+   positives, phishing->spam and spam->phishing confusions with previews, lexical
+   patterns, written summary), ``metrics.json`` (with a ``threeway`` block), ONE line in
+   ``experiments.jsonl`` and ONE RUN_LOG.md entry per invocation covering every split
+   that was run (skipped for ``--limit-rows`` dev runs, which are not evidence).
+   ``reports/pretrain_curves.png`` and ``finetune_curves.png`` are written by
+   ``pretrain.py plot`` / ``finetune.py plot`` and are never touched here;
+   ``--finetune-run-dir`` / ``--pretrain-run-dir`` opt into extra ``eval_*_curves.png``.
+
+Outputs for ``--split val`` get a ``_val`` suffix and for ``--split test3`` a ``_test3``
+suffix so no split's report overwrites another's. ``--out-dir`` redirects every file, so a
+self-test never writes into reports/; ``--data-dir`` points at fixture parquet files.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import logging
+import math
+import re
+import sys
+import timeit
+import unicodedata
+from collections import Counter
+from dataclasses import dataclass
+from datetime import datetime
+from pathlib import Path
+from typing import Any, Sequence
+
+sys.path.insert(0, str(Path(__file__).parent))
+
+import matplotlib
+
+matplotlib.use("Agg")
+import matplotlib.pyplot as plt  # noqa: E402
+import numpy as np  # noqa: E402
+import pyarrow.parquet as pq  # noqa: E402
+import torch  # noqa: E402
+from safetensors.torch import load_file  # noqa: E402
+from sklearn.metrics import (  # noqa: E402
+    ConfusionMatrixDisplay,
+    average_precision_score,
+    confusion_matrix,
+    precision_recall_curve,
+    precision_recall_fscore_support,
+    roc_auc_score,
+    roc_curve,
+)
+
+import common  # noqa: E402
+from model import EncoderConfig, EncoderForClassification, count_parameters  # noqa: E402
+from tokenizer import encode_batch, load_tokenizer  # noqa: E402
+
+LOG = logging.getLogger("evaluate")
+
+SPLIT_FILES: dict[str, str] = {"test": "test_holdout.parquet", "val": "finetune_val.parquet", "test3": "test_holdout3.parquet"}
+SPLIT_SUFFIX: dict[str, str] = {"test": "", "val": "_val", "test3": "_test3"}
+SPLIT_TITLE: dict[str, str] = {"test": "TREC-06 binary holdout", "val": "validation split", "test3": "three-way holdout"}
+# Three-way label contract (RUN_LOG entry 6): 0 legitimate, 1 phishing, 2 spam; -1 = unwanted-but-unsplit
+# (TREC-05 / TREC-07 / CEAS-08 positives), which only enter the binary roll-up.
+LABEL_NAMES: dict[int, str] = {0: "Legitimate", 1: "Phishing", 2: "Spam"}
+UNWANTED_NAME = "Unwanted (phishing or spam)"
+BINARY_NAMES: dict[int, str] = {0: "Legitimate", 1: UNWANTED_NAME}
+LABEL_SET_OF: dict[int, str] = {0: "legit", 1: "phish", 2: "spam", -1: "unwanted"}
+DECISION_RULE = "rollup_then_argmax"
+DECISION_RULE_DOC = ("unwanted iff P(phishing)+P(spam) >= threshold, then kind = argmax(P(phishing), P(spam)); "
+                     "the 3x3 matrix therefore collapses exactly onto the binary confusion matrix. argmax-of-3 reported alongside.")
+INSTRUCTOR_BENCHMARK_MS = 74.0
+DEFAULT_THRESHOLD = 0.5
+CPU_LATENCY_BUDGET_S = 60.0
+CPU_LATENCY_FALLBACK_N = 10
+TOP_K_ERRORS = 25
+TOP_K_CONFUSIONS = 25
+PREVIEW_CHARS = 160
+ENCODE_BATCH_ROWS = 2048
+LENGTH_BUCKETS: tuple[tuple[int, int, str], ...] = (
+    (0, 64, "<=64"), (65, 128, "65-128"), (129, 256, "129-256"), (257, 512, "257-512"), (513, 10**9, ">512"),
+)
+
+# Credential-lure phrases. A row belongs to the slice when the phrase occurs as whole
+# words in the lure-normalised text (NFKC -> lowercase -> every non-alphanumeric run
+# becomes one space), so casing, punctuation, HTML tags and line breaks cannot hide a
+# match ("Verify  your<br>account" still hits) while word boundaries hold: "urgent"
+# no longer matches "insurgent"/"resurgent" (the alnum-only match did), and "urgently"
+# is listed separately because urgency is the common denominator of credential lures.
+# The list is documented here rather than tuned on the test set: it is the subset of
+# the positive class whose cost of a miss is highest (account takeover / payment
+# fraud). Phrases that were majority-legitimate on finetune_val (limited access, credit
+# card number, billing/payment information, social security number) were dropped so
+# the slice's "precision" says something about the model rather than about the slice
+# definition; the slice's own label purity is printed next to its metrics for the same
+# reason.
+CREDENTIAL_LURE_PHRASES: tuple[str, ...] = (
+    "verify your account", "verify your identity", "confirm your identity", "confirm your account",
+    "enter your password", "update your account", "update your information", "update your billing",
+    "update your payment", "click here to verify", "click the link below to verify",
+    "account has been suspended", "account has been limited", "account will be suspended",
+    "account has been locked", "unusual sign-in", "unusual activity", "suspicious activity",
+    "paypal account", "ebay account", "password expire", "password will expire", "reset your password",
+    "invoice attached", "wire transfer", "gift card", "urgent", "urgently", "security alert", "security notice",
+    "login to your account", "log in to your account", "sign in to your account",
+    "bank account", "failure to verify", "within 24 hours", "within 48 hours",
+)
+LURE_MIN_POSITIVES = 20   # below this the slice recall is reported but not compared to the overall recall
+_LURE_NON_ALNUM = re.compile(r"[^a-z0-9]+")
+
+# Palette: dataviz reference instance (slot 1 blue, 2 orange, 3 aqua; ink; surface).
+C_BLUE, C_ORANGE, C_AQUA, C_VIOLET = "#2a78d6", "#eb6834", "#1baf7a", "#4a3aa7"
+C_INK, C_INK2, C_GRID, C_SURFACE = "#0b0b0b", "#52514e", "#dddcd7", "#fcfcfb"
+PLOT_DPI = 150
+
+
+# ------------------------------------------------------------------------------- data
+@dataclass
+class Split:
+    name: str
+    path: Path
+    doc_id: list[str]
+    text: list[str]
+    subject: list[str]
+    body: list[str]
+    label: np.ndarray        # binary: 1 = unwanted (phishing, spam or unsplit positive)
+    label3: np.ndarray       # 0 legitimate / 1 phishing / 2 spam / -1 unwanted-but-unsplit
+    label_set: list[str]     # legit / phish / spam / unwanted
+    source: list[str]
+    limit_rows: int | None
+
+    @property
+    def n(self) -> int:
+        return len(self.text)
+
+    @property
+    def sfx(self) -> str:
+        return SPLIT_SUFFIX[self.name]
+
+
+def check_eval_config(cfg: dict) -> int:
+    """Reconcile the documented ``eval:`` dials with this module's constants and return ``top_k_confusions``.
+
+    ``label_names``, ``unwanted_name`` and ``threeway_decision_rule`` are the three-way label
+    contract shared with finetune.py, export.py and the app, so they are fixed in code and
+    config.yaml may only restate them: a drift raises here, loudly, instead of producing a card
+    whose names differ from the metrics. ``top_k_confusions`` is a real dial (size of the
+    phishing->spam / spam->phishing preview tables). ``eval.metrics_threeway`` is documentation
+    only: it lists the sections the three-way block always contains."""
+    ev = cfg.get("eval") or {}
+    names = ev.get("label_names")
+    if names is not None:
+        got = {int(c): str(v) for c, v in dict(names).items()}
+        if got != LABEL_NAMES:
+            raise ValueError(f"config eval.label_names {got} != evaluate.LABEL_NAMES {LABEL_NAMES}: the label contract is fixed in code")
+    if ev.get("unwanted_name") is not None and str(ev["unwanted_name"]) != UNWANTED_NAME:
+        raise ValueError(f"config eval.unwanted_name {ev['unwanted_name']!r} != {UNWANTED_NAME!r}: fixed in code")
+    if ev.get("threeway_decision_rule") is not None and str(ev["threeway_decision_rule"]) != DECISION_RULE:
+        raise ValueError(f"config eval.threeway_decision_rule {ev['threeway_decision_rule']!r} != {DECISION_RULE!r}: only this rule is implemented")
+    top_k = int(ev.get("top_k_confusions", TOP_K_CONFUSIONS))
+    if top_k < 1:
+        raise ValueError(f"config eval.top_k_confusions must be >= 1, got {top_k}")
+    return top_k
+
+
+def split_path(data_dir: Path, name: str, test3_file: str) -> Path:
+    return data_dir / (test3_file if name == "test3" else SPLIT_FILES[name])
+
+
+def load_split(data_dir: Path, name: str, limit_rows: int | None, seed: int, test3_file: str = SPLIT_FILES["test3"]) -> Split:
+    """Read one labelled split. ``limit_rows`` takes a seeded random sample (dev only);
+    the full run reads every row so the report is the whole holdout.
+
+    ``label3`` / ``label_set`` are read when the file carries them (data_prep writes them
+    for every split since RUN_LOG entry 6). A binary-era file without them is still
+    evaluable: every positive is an unsplit "unwanted" row (-1) and every negative is
+    legitimate (0), which is precisely the TREC-06 contract, so the binary roll-up is
+    unaffected and the three-way section simply has no rows."""
+    path = split_path(data_dir, name, test3_file)
+    if not path.exists():
+        raise FileNotFoundError(path)
+    have = set(pq.read_schema(path).names)
+    cols = ["doc_id", "text", "subject", "body", "source"] + [c for c in ("label", "label3", "label_set") if c in have]
+    if "label" not in have and "label3" not in have:
+        raise ValueError(f"{path}: neither 'label' nor 'label3' column present")
+    table = pq.read_table(path, columns=cols)
+    if limit_rows is not None and table.num_rows > limit_rows:
+        idx = np.sort(np.random.default_rng(seed).choice(table.num_rows, size=limit_rows, replace=False))
+        table = table.take(idx)
+    d = table.to_pydict()
+    n = table.num_rows
+    if "label3" in have:
+        label3 = np.asarray(d["label3"], dtype=np.int64)
+        bad = sorted(set(label3.tolist()) - set(LABEL_SET_OF))
+        if bad:
+            raise ValueError(f"{path}: label3 values outside {{-1,0,1,2}}: {bad}")
+    else:
+        label3 = None
+    if "label" in have:
+        label = np.asarray(d["label"], dtype=np.int64)
+    else:
+        label = (label3 != 0).astype(np.int64)
+    if label3 is None:
+        label3 = np.where(label == 1, -1, 0).astype(np.int64)
+        LOG.info("%s has no label3 column: positives treated as unwanted-but-unsplit (-1), negatives as legitimate (0)", path.name)
+    if "label_set" in have:
+        label_set = [str(s) for s in d["label_set"]]
+        # label_set is the human-readable twin of label3 (finetune.read_split enforces the same
+        # pairing); a disagreement means data_prep wrote inconsistent columns, and the error
+        # tables below print label_set, so refuse rather than report the wrong class name.
+        mismatch = [i for i in range(n) if label_set[i] != LABEL_SET_OF[int(label3[i])]]
+        if mismatch:
+            i = mismatch[0]
+            raise ValueError(f"{path}: 'label_set' disagrees with 'label3' on {len(mismatch)} row(s) (first: row {i} "
+                             f"label_set={label_set[i]!r} but label3={int(label3[i])} implies {LABEL_SET_OF[int(label3[i])]!r}); "
+                             f"contract: {LABEL_SET_OF}")
+    else:
+        label_set = [LABEL_SET_OF[int(v)] for v in label3]
+    if bool(((label3 != 0).astype(np.int64) != label).any()):
+        raise ValueError(f"{path}: 'label' disagrees with 'label3 != 0' on some rows (contract: label = 1 for phish/spam/unwanted)")
+    return Split(
+        name=name, path=path,
+        doc_id=[str(x) for x in d["doc_id"]],
+        text=[t or "" for t in d["text"]],
+        subject=[s or "" for s in d["subject"]],
+        body=[b or "" for b in d["body"]],
+        label=label, label3=label3, label_set=label_set,
+        source=[str(s) for s in d["source"]],
+        limit_rows=limit_rows,
+    )
+
+
+@dataclass
+class Encoded:
+    """Per-row token ids as the model sees them, plus the untruncated length for the confound check."""
+
+    ids: list[list[int]]
+    n_tokens: np.ndarray        # fed to the model (incl. [CLS]/[SEP]), <= max_len
+    n_tokens_full: np.ndarray   # before truncation
+    truncated: np.ndarray
+
+
+def encode_split(tok, texts: Sequence[str], max_len: int) -> Encoded:
+    """Encode once without truncation (so the full length is known), then truncate the
+    content to ``max_len - 2`` and keep ``[CLS] ... [SEP]``, which is exactly what
+    ``tokenizers`` truncation + the BERT template produce."""
+    tok.no_truncation()
+    tok.no_padding()
+    sep_id = tok.token_to_id("[SEP]")
+    ids: list[list[int]] = []
+    full = np.empty(len(texts), dtype=np.int64)
+    for i in range(0, len(texts), ENCODE_BATCH_ROWS):
+        for j, enc in enumerate(tok.encode_batch_fast(list(texts[i:i + ENCODE_BATCH_ROWS]))):
+            seq = enc.ids
+            full[i + j] = len(seq)
+            if len(seq) > max_len:
+                seq = seq[: max_len - 1] + [sep_id]
+            ids.append(seq)
+    n_tok = np.minimum(full, max_len)
+    return Encoded(ids, n_tok, full, full > max_len)
+
+
+# ------------------------------------------------------------------------------ model
+def load_classifier(ckpt_dir: Path, device: torch.device) -> tuple[EncoderForClassification, EncoderConfig, dict]:
+    """Rebuild the classifier from ``config.json["model"]`` (including ``num_labels``) and
+    load ``model.safetensors`` strictly.
+
+    Strict loading is non-negotiable: a half-loaded encoder still produces confident
+    probabilities, and nothing downstream could tell the difference. A head whose shape
+    disagrees with ``num_labels`` fails here, loudly. Weights may be fp32 (training
+    checkpoint) or bf16 (export); ``load_state_dict`` casts into the fp32 module, and the
+    GPU path runs under bf16 autocast either way.
+    """
+    cfg_json = common.read_json(ckpt_dir / "config.json")
+    mcfg = EncoderConfig.from_dict(cfg_json["model"])
+    mcfg.grad_checkpointing = False
+    if mcfg.num_labels not in (2, 3):
+        raise ValueError(f"num_labels={mcfg.num_labels}: this evaluator understands 2 (binary) or 3 (legitimate/phishing/spam)")
+    with torch.device(device):
+        model = EncoderForClassification(mcfg)
+    sd = load_file(str(ckpt_dir / "model.safetensors"), device="cpu")
+    model.load_state_dict(sd, strict=True)
+    model.eval()
+    dtypes = sorted({str(v.dtype).replace("torch.", "") for v in sd.values()})
+    LOG.info("loaded %s: %d tensors (%s), %s params, num_labels=%d", ckpt_dir / "model.safetensors", len(sd), dtypes,
+             f"{count_parameters(model)['total']:,}", mcfg.num_labels)
+    return model, mcfg, cfg_json
+
+
+def _pad_batch(enc: Encoded, idx: np.ndarray, multiple: int) -> tuple[torch.Tensor, torch.Tensor]:
+    L = int(enc.n_tokens[idx].max())
+    L = int(math.ceil(L / multiple) * multiple)
+    arr = np.zeros((len(idx), L), dtype=np.int64)
+    mask = np.zeros((len(idx), L), dtype=np.bool_)
+    for r, i in enumerate(idx):
+        seq = enc.ids[i]
+        arr[r, : len(seq)] = seq
+        mask[r, : len(seq)] = True
+    return torch.from_numpy(arr), torch.from_numpy(mask)
+
+
+@torch.inference_mode()
+def predict_probs(model: EncoderForClassification, enc: Encoded, device: torch.device,
+                  batch_size: int, pad_multiple: int) -> tuple[np.ndarray, dict[str, Any]]:
+    """Full softmax ``[n, num_labels]`` per row. Rows are sorted longest-first so each batch
+    pads to its own longest member (dynamic padding; ~2x fewer pad tokens than padding to
+    512) and an OOM would surface on the first batch rather than after ten minutes."""
+    n = len(enc.ids)
+    k = int(model.config.num_labels)
+    order = np.argsort(-enc.n_tokens, kind="stable")
+    probs = np.empty((n, k), dtype=np.float32)
+    use_amp = device.type == "cuda"
+    padded_tokens = 0
+    n_batches = math.ceil(n / batch_size)
+    t0 = common.timer()
+    for b, start in enumerate(range(0, n, batch_size)):
+        idx = order[start: start + batch_size]
+        ids, mask = _pad_batch(enc, idx, pad_multiple)
+        padded_tokens += ids.numel()
+        ids, mask = ids.to(device, non_blocking=True), mask.to(device, non_blocking=True)
+        with torch.autocast(device_type="cuda", dtype=torch.bfloat16, enabled=use_amp):
+            logits = model(ids, mask)
+        probs[idx] = logits.float().softmax(-1).cpu().numpy()
+        if b % 20 == 0 or b == n_batches - 1:
+            LOG.info("inference batch %d/%d (L=%d) %.1fs", b + 1, n_batches, ids.shape[1], common.timer() - t0)
+    if use_amp:
+        torch.cuda.synchronize()
+    secs = common.timer() - t0
+    info = {
+        "device": str(device), "autocast_bf16": use_amp, "batch_size": batch_size, "pad_to_multiple_of": pad_multiple,
+        "num_labels": k, "rows": n, "seconds": round(secs, 2), "rows_per_s": round(n / secs, 1),
+        "padded_tokens": int(padded_tokens), "real_tokens": int(enc.n_tokens.sum()),
+        "padded_tokens_per_s": round(padded_tokens / secs),
+        "peak_vram_gb": round(common.vram_stats()["max_allocated_gb"], 2),
+    }
+    LOG.info("inference: %s", info)
+    return probs, info
+
+
+def p_unwanted(probs: np.ndarray) -> np.ndarray:
+    """Binary roll-up: P(unwanted) = sum of every non-legitimate class. For a 2-class head
+    this is exactly P(class 1), so the binary report is byte-for-byte the old definition."""
+    return np.clip(probs[:, 1:].sum(axis=1), 0.0, 1.0).astype(np.float32)
+
+
+# ---------------------------------------------------------------------------- metrics
+def _safe_div(a: float, b: float) -> float:
+    return float(a) / float(b) if b else 0.0
+
+
+def threshold_metrics(y: np.ndarray, p: np.ndarray, thr: float) -> dict[str, Any]:
+    """Confusion-matrix metrics at ``p >= thr``. An empty slice (e.g. no credential-lure
+    rows in a small ``--limit-rows`` sample) returns all-zero counts instead of hitting
+    sklearn's minimum-one-sample check."""
+    if len(y) == 0:
+        tn = fp = fn = tp = 0
+    else:
+        pred = (p >= thr).astype(np.int64)
+        cm = confusion_matrix(y, pred, labels=[0, 1])
+        tn, fp, fn, tp = (int(v) for v in cm.ravel())
+    precision = _safe_div(tp, tp + fp)
+    recall = _safe_div(tp, tp + fn)
+    return {
+        "threshold": thr,
+        "accuracy": _safe_div(tp + tn, len(y)),
+        "precision": precision,
+        "recall": recall,
+        "f1": _safe_div(2 * precision * recall, precision + recall),
+        "specificity": _safe_div(tn, tn + fp),
+        "false_positive_rate": _safe_div(fp, tn + fp),
+        "false_negative_rate": _safe_div(fn, tp + fn),
+        "confusion_matrix": {"tn": tn, "fp": fp, "fn": fn, "tp": tp},
+        "n": int(len(y)), "n_positive": int(y.sum()), "n_negative": int(len(y) - y.sum()),
+    }
+
+
+def _both_classes(y: np.ndarray) -> bool:
+    return bool(y.min() == 0 and y.max() == 1) if len(y) else False
+
+
+def threshold_free_metrics(y: np.ndarray, p: np.ndarray) -> dict[str, Any]:
+    """ROC-AUC, PR-AUC (average precision), recall at a fixed FPR budget read off the ROC
+    curve, and the threshold that maximises F1. The FPR-budget numbers are the ones an
+    inbox operator actually asks ("if I may bother users with 1 false alarm in 100
+    legitimate emails, how many phish do I catch?"). Thresholds are to be applied as
+    ``p >= threshold``; when the only point inside the budget is the ROC origin, sklearn's
+    sentinel threshold is +inf and ``threshold`` is reported as None (no finite score
+    cut reaches that budget) rather than clamped to 1.0, which bf16 probabilities that
+    round to exactly 1.0 would still pass."""
+    if not _both_classes(y):
+        LOG.warning("only one class present: ROC/PR metrics undefined")
+        return {"roc_auc": None, "pr_auc": None, "recall_at_fpr_1pct": None, "recall_at_fpr_0_1pct": None,
+                "best_f1": None, "curves": None}
+    fpr, tpr, roc_thr = roc_curve(y, p)
+    prec, rec, pr_thr = precision_recall_curve(y, p)
+
+    def recall_at_fpr(budget: float) -> dict[str, float]:
+        ok = np.nonzero(fpr <= budget)[0]
+        i = ok[np.argmax(tpr[ok])]
+        t = float(roc_thr[i])
+        return {"fpr_budget": budget, "recall": float(tpr[i]), "fpr": float(fpr[i]),
+                "threshold": t if math.isfinite(t) else None, "threshold_rule": "flag when p >= threshold"}
+
+    f1 = 2 * prec[:-1] * rec[:-1] / np.clip(prec[:-1] + rec[:-1], 1e-12, None)
+    j = int(np.argmax(f1))
+    return {
+        "roc_auc": float(roc_auc_score(y, p)),
+        "pr_auc": float(average_precision_score(y, p)),
+        "recall_at_fpr_1pct": recall_at_fpr(0.01),
+        "recall_at_fpr_0_1pct": recall_at_fpr(0.001),
+        "best_f1": {"threshold": float(pr_thr[j]), "f1": float(f1[j]), "precision": float(prec[j]), "recall": float(rec[j])},
+        "curves": {"fpr": fpr, "tpr": tpr, "precision": prec, "recall": rec},
+    }
+
+
+def lure_normalize(s: str) -> str:
+    """Word-boundary-preserving sibling of ``common.normalize_text``: HTML tags removed,
+    NFKC -> lowercase -> every non-alphanumeric run collapses to one space, padded with a
+    space on both ends so a phrase match ``" ph "`` is a whole-word match
+    ("Verify your<br>account" -> " verify your account ")."""
+    s = _HTML_TAG.sub(" ", s or "")
+    return " " + _LURE_NON_ALNUM.sub(" ", unicodedata.normalize("NFKC", s).lower()).strip() + " "
+
+
+def lure_mask(texts: Sequence[str], labels: np.ndarray | None = None) -> tuple[np.ndarray, Counter, dict[str, dict[str, int]]]:
+    """Rows containing any lure phrase as whole words, per-phrase hit counts, and (when
+    labels are given) per-phrase positive/negative hits so the slice's purity is visible."""
+    norm_phrases = [(ph, lure_normalize(ph)) for ph in CREDENTIAL_LURE_PHRASES]
+    hits = np.zeros(len(texts), dtype=np.bool_)
+    per_phrase: Counter = Counter()
+    by_label: dict[str, dict[str, int]] = {ph: {"positive": 0, "negative": 0} for ph in CREDENTIAL_LURE_PHRASES}
+    for i, t in enumerate(texts):
+        n = lure_normalize(t)
+        for ph, nph in norm_phrases:
+            if nph in n:
+                hits[i] = True
+                per_phrase[ph] += 1
+                if labels is not None:
+                    by_label[ph]["positive" if int(labels[i]) == 1 else "negative"] += 1
+    return hits, per_phrase, {ph: c for ph, c in by_label.items() if per_phrase[ph]}
+
+
+def slice_metrics(y: np.ndarray, p: np.ndarray, thr: float) -> dict[str, Any]:
+    m = threshold_metrics(y, p, thr)
+    m["roc_auc"] = float(roc_auc_score(y, p)) if _both_classes(y) else None
+    return m
+
+
+def per_source_table(split: Split, p: np.ndarray, thr: float) -> dict[str, dict[str, Any]]:
+    out: dict[str, dict[str, Any]] = {}
+    src = np.asarray(split.source)
+    for s in sorted(set(split.source)):
+        sel = src == s
+        m = slice_metrics(split.label[sel], p[sel], thr)
+        m["label3_counts"] = _label3_counts(split.label3[sel])
+        out[s] = m
+    return out
+
+
+def _label3_counts(label3: np.ndarray) -> dict[str, int]:
+    return {LABEL_SET_OF[k]: int((label3 == k).sum()) for k in (0, 1, 2, -1)}
+
+
+def length_confound(split: Split, enc: Encoded, p: np.ndarray, thr: float) -> dict[str, Any]:
+    """Mean token length of FP/FN vs correct rows, by label, and recall/FPR per length
+    bucket. If errors concentrate in the length range where the classes' length
+    distributions overlap (long positives, short legitimate), the model is at least
+    partly reading length rather than content. ``by_class3`` adds the phishing / spam /
+    unsplit split of the positive class so a phishing-vs-spam length gap is visible too."""
+    y, pred = split.label, (p >= thr).astype(np.int64)
+    groups = {"tp": (y == 1) & (pred == 1), "tn": (y == 0) & (pred == 0),
+              "fp": (y == 0) & (pred == 1), "fn": (y == 1) & (pred == 0)}
+
+    def stats(sel: np.ndarray) -> dict[str, Any]:
+        if not sel.any():
+            return {"n": 0, "mean_tokens_full": None, "mean_tokens_fed": None, "median_tokens_full": None, "pct_truncated": None}
+        return {"n": int(sel.sum()), "mean_tokens_full": round(float(enc.n_tokens_full[sel].mean()), 1),
+                "mean_tokens_fed": round(float(enc.n_tokens[sel].mean()), 1),
+                "median_tokens_full": float(np.median(enc.n_tokens_full[sel])),
+                "pct_truncated": round(100 * float(enc.truncated[sel].mean()), 1)}
+
+    by_outcome = {k: stats(v) for k, v in groups.items()}
+    by_outcome["correct"] = stats(groups["tp"] | groups["tn"])
+    by_outcome["wrong"] = stats(groups["fp"] | groups["fn"])
+    by_label = {"legitimate": stats(y == 0), "unwanted": stats(y == 1)}
+    by_class3 = {LABEL_SET_OF[k]: stats(split.label3 == k) for k in (0, 1, 2, -1) if (split.label3 == k).any()}
+    buckets = []
+    for lo, hi, name in LENGTH_BUCKETS:
+        sel = (enc.n_tokens_full >= lo) & (enc.n_tokens_full <= hi)
+        if not sel.any():
+            continue
+        m = threshold_metrics(y[sel], p[sel], thr)
+        buckets.append({"bucket": name, "n": m["n"], "n_positive": m["n_positive"], "positive_rate": _safe_div(m["n_positive"], m["n"]),
+                        "recall": m["recall"] if m["n_positive"] else None,
+                        "fpr": m["false_positive_rate"] if m["n_negative"] else None, "accuracy": m["accuracy"]})
+    return {"by_outcome": by_outcome, "by_label": by_label, "by_class3": by_class3, "by_length_bucket": buckets}
+
+
+# ------------------------------------------------------------------------- three-way
+@dataclass
+class ThreeWay:
+    """Three-way result for one split. ``metrics`` is the JSON block; ``pred3`` / ``kind``
+    are per-row arrays (full length) that the error analysis uses, None for a 2-class head."""
+
+    evaluated: bool
+    metrics: dict[str, Any]
+    pred3: np.ndarray | None
+    kind: np.ndarray | None
+
+
+def predict3(probs: np.ndarray, p_unw: np.ndarray, thr: float) -> tuple[np.ndarray, np.ndarray]:
+    """Roll-up-first decision: ``kind`` = argmax over the unwanted classes (1 phishing /
+    2 spam) for every row; ``pred3`` = kind when P(unwanted) >= thr else 0 (legitimate).
+    Choosing the kind *conditionally* on being unwanted (rather than argmax-of-3) keeps the
+    three-way prediction consistent with the binary verdict the app shows: a row the app
+    flags can never be "legitimate" in the 3x3 table, and vice versa."""
+    kind = 1 + probs[:, 1:3].argmax(axis=1)
+    pred3 = np.where(p_unw >= thr, kind, 0).astype(np.int64)
+    return pred3, kind.astype(np.int64)
+
+
+def _cm_dict(mat: np.ndarray, labels: Sequence[int]) -> dict[str, dict[str, int]]:
+    return {f"true_{LABEL_NAMES[a].lower()}": {f"pred_{LABEL_NAMES[b].lower()}": int(mat[i, j]) for j, b in enumerate(labels)}
+            for i, a in enumerate(labels)}
+
+
+def _macro_block(y: np.ndarray, pred: np.ndarray) -> dict[str, Any]:
+    p, r, f, s = precision_recall_fscore_support(y, pred, labels=[0, 1, 2], zero_division=0)
+    per_class = {LABEL_NAMES[k]: {"precision": float(p[k]), "recall": float(r[k]), "f1": float(f[k]), "support": int(s[k])} for k in range(3)}
+    present = [k for k in range(3) if s[k] > 0]
+    return {"accuracy": float((y == pred).mean()) if len(y) else 0.0,
+            "macro_f1": float(np.mean([f[k] for k in range(3)])),
+            "macro_f1_present_classes": float(np.mean([f[k] for k in present])) if present else 0.0,
+            "classes_present": [LABEL_NAMES[k] for k in present],
+            "per_class": per_class}
+
+
+def threeway_metrics(split: Split, probs: np.ndarray, p_unw: np.ndarray, thr: float) -> ThreeWay:
+    """Three-way block for one split.
+
+    * 2-class head: nothing to evaluate (the checkpoint has no phishing-vs-spam output).
+    * 3-class head: ``predicted_kind_breakdown`` is ALWAYS written (how the flagged rows
+      split into predicted phishing vs spam); it is informational on TREC-06, where every
+      positive is an unsplit "unwanted" row and no ground truth exists, and it says so.
+    * rows with ``label3`` in {0, 1, 2}: 3x3 confusion matrix, per-class P/R/F1, macro-F1
+      (over all three classes, zero F1 for an absent class, plus the mean over the classes
+      actually present), argmax-of-3 as a secondary rule, a phishing-vs-spam confusion on
+      the true unwanted rows (the kind decision alone, i.e. "given it is unwanted, which
+      kind", plus the roll-up view that also shows how many were missed as legitimate),
+      and per-source class counts / accuracy / macro-F1.
+    """
+    k = probs.shape[1]
+    base: dict[str, Any] = {"num_labels": k, "label_names": {str(c): LABEL_NAMES[c] for c in range(k) if c in LABEL_NAMES},
+                            "unwanted_name": UNWANTED_NAME, "decision_rule": DECISION_RULE, "decision_rule_doc": DECISION_RULE_DOC,
+                            "threshold": thr, "label3_counts": _label3_counts(split.label3)}
+    if k != 3:
+        base["label_names"] = {"0": BINARY_NAMES[0], "1": BINARY_NAMES[1]}
+        return ThreeWay(False, {**base, "evaluated": False, "reason": f"{k}-class checkpoint: no phishing-vs-spam head, three-way sections skipped"}, None, None)
+    pred3, kind = predict3(probs, p_unw, thr)
+    argmax3 = probs.argmax(axis=1)
+    y, y3 = split.label, split.label3
+    flagged = p_unw >= thr
+
+    def kinds(sel: np.ndarray) -> dict[str, int]:
+        return {"n": int(sel.sum()), "predicted_phishing": int((kind[sel] == 1).sum()), "predicted_spam": int((kind[sel] == 2).sum())}
+
+    base["predicted_kind_breakdown"] = {
+        "note": "How the model's flagged rows split into predicted phishing vs spam (kind = argmax(P(phishing), P(spam))). "
+                "Informational: for rows labelled -1/unwanted (all TREC-06 positives) there is NO phishing-vs-spam ground truth.",
+        "true_positives_flagged": kinds((y == 1) & flagged), "false_positives_flagged": kinds((y == 0) & flagged),
+        "missed_positives": kinds((y == 1) & ~flagged), "all_flagged": kinds(flagged),
+        "unsplit_unwanted_rows_flagged": kinds((y3 == -1) & flagged),
+    }
+    full = (y3 >= 0) & (y3 <= 2)
+    base["rows_with_full_label"] = int(full.sum())
+    base["rows_unwanted_only"] = int((y3 == -1).sum())
+    # Legitimate rows alone (label3 == 0) say nothing about phishing-vs-spam: a split whose every
+    # positive is unsplit (TREC-06) would otherwise yield a degenerate 3x3 with zero phishing/spam
+    # support and a meaningless macro-F1. At least one row labelled phishing or spam is required.
+    if not (full & (y3 >= 1)).any():
+        return ThreeWay(False, {**base, "evaluated": False,
+                                "reason": "no rows labelled phishing or spam: every positive is unwanted-but-unsplit (no phishing-vs-spam ground truth)"},
+                        pred3, kind)
+    yf, pf, af = y3[full], pred3[full], argmax3[full]
+    mat = confusion_matrix(yf, pf, labels=[0, 1, 2])
+    main = _macro_block(yf, pf)
+    alt = _macro_block(yf, af)
+    tu = full & (y3 >= 1)
+    if tu.any():
+        kmat = confusion_matrix(y3[tu], kind[tu], labels=[1, 2])
+        rmat = confusion_matrix(y3[tu], pred3[tu], labels=[0, 1, 2])[1:, :]
+        pvs = {"n_true_unwanted": int(tu.sum()),
+               "kind_only": {"doc": "given the row is unwanted, is argmax(P(phishing), P(spam)) right? (independent of the roll-up threshold)",
+                             "labels": ["Phishing", "Spam"], "matrix": kmat.tolist(), "cells": _cm_dict(kmat, [1, 2]),
+                             "accuracy": float(np.trace(kmat) / max(kmat.sum(), 1)),
+                             "phishing_as_spam": int(kmat[0, 1]), "spam_as_phishing": int(kmat[1, 0])},
+               "rollup_rule": {"doc": "rows = true Phishing / Spam; columns = predicted Legitimate (missed) / Phishing / Spam under the roll-up rule",
+                               "rows": ["Phishing", "Spam"], "columns": ["Legitimate", "Phishing", "Spam"], "matrix": rmat.tolist(),
+                               "missed_as_legitimate": {"phishing": int(rmat[0, 0]), "spam": int(rmat[1, 0])}}}
+    else:
+        pvs = {"n_true_unwanted": 0, "kind_only": None, "rollup_rule": None}
+    src = np.asarray(split.source)
+    per_source: dict[str, Any] = {}
+    for s in sorted(set(split.source)):
+        sel = full & (src == s)
+        if not sel.any():
+            continue
+        b = _macro_block(y3[sel], pred3[sel])
+        per_source[s] = {"n": int(sel.sum()), "counts": {LABEL_NAMES[c]: int((y3[sel] == c).sum()) for c in range(3)},
+                         "accuracy": b["accuracy"], "macro_f1": b["macro_f1"], "macro_f1_present_classes": b["macro_f1_present_classes"],
+                         "per_class_recall": {c: v["recall"] for c, v in b["per_class"].items() if v["support"]}}
+    metrics = {**base, "evaluated": True, "reason": None,
+               "confusion_matrix": {"labels": [LABEL_NAMES[c] for c in range(3)], "matrix": mat.tolist(), "cells": _cm_dict(mat, [0, 1, 2])},
+               "accuracy": main["accuracy"], "macro_f1": main["macro_f1"], "macro_f1_present_classes": main["macro_f1_present_classes"],
+               "classes_present": main["classes_present"], "per_class": main["per_class"],
+               "argmax_rule": {"doc": "plain argmax over the three probabilities (secondary; can disagree with the binary verdict near the threshold)",
+                               "accuracy": alt["accuracy"], "macro_f1": alt["macro_f1"], "per_class": alt["per_class"],
+                               "rows_where_rules_differ": int((pf != af).sum())},
+               "phish_vs_spam_on_true_unwanted": pvs, "per_source": per_source}
+    return ThreeWay(True, metrics, pred3, kind)
+
+
+# ---------------------------------------------------------------------------- latency
+def _time_single(model, tok, texts: Sequence[str], device: torch.device, max_len: int,
+                 use_amp: bool, warmup: int = 3) -> list[float]:
+    """Wall-clock ms per email for tokenize + forward + softmax at batch 1, measured with
+    ``timeit.default_timer`` exactly like the instructor's step-5 inference timer.
+    ``.item()`` forces the GPU sync so the stop time includes the kernel work (the
+    roll-up sum is part of what the app computes per request, so it is timed too)."""
+    times: list[float] = []
+    with torch.inference_mode():
+        for k, t in enumerate(texts):
+            t0 = timeit.default_timer()
+            ids, mask = encode_batch(tok, [t], max_len)
+            ids, mask = ids.to(device), mask.to(device)
+            with torch.autocast(device_type="cuda", dtype=torch.bfloat16, enabled=use_amp):
+                logits = model(ids, mask)
+            _ = logits.float().softmax(-1)[0, 1:].sum().item()
+            if k >= warmup:
+                times.append((timeit.default_timer() - t0) * 1000.0)
+    return times
+
+
+def _summ(times: list[float], n_tokens: Sequence[int]) -> dict[str, Any]:
+    arr = np.asarray(times)
+    return {"n": int(len(arr)), "avg_ms": round(float(arr.mean()), 2), "p50_ms": round(float(np.median(arr)), 2),
+            "p95_ms": round(float(np.percentile(arr, 95)), 2), "max_ms": round(float(arr.max()), 2),
+            "avg_s": round(float(arr.mean()) / 1000.0, 4), "mean_tokens": round(float(np.mean(n_tokens)), 1),
+            "ratio_to_benchmark": round(float(arr.mean()) / INSTRUCTOR_BENCHMARK_MS, 3)}
+
+
+def measure_latency(model, tok, split: Split, enc: Encoded, device: torch.device, max_len: int,
+                    n_samples: int, cpu_threads: int, seed: int) -> dict[str, Any]:
+    """GPU bf16 then CPU fp32 on the same seeded sample of real emails. The CPU pass is
+    cut to 10 samples when 3 warmups predict more than 60 s for the full count (the
+    smoke test measured ~0.7 s per 512-token email on 16 threads). Moves the model to
+    the CPU, so it is the last thing an invocation does with the model."""
+    rng = np.random.default_rng(seed)
+    n = min(n_samples, split.n)
+    idx = rng.choice(split.n, size=n, replace=False)
+    texts = [split.text[i] for i in idx]
+    toks = [int(enc.n_tokens[i]) for i in idx]
+    out: dict[str, Any] = {"instructor_benchmark_ms": INSTRUCTOR_BENCHMARK_MS, "samples_requested": n_samples,
+                           "sample_split": split.name, "max_len": max_len}
+    warm = texts[:3] + texts  # 3 warmups, then every sample timed
+    if device.type == "cuda":
+        torch.cuda.synchronize()
+        gpu_times = _time_single(model, tok, warm, device, max_len, use_amp=True)
+        out["gpu_bf16"] = _summ(gpu_times, toks)
+        out["gpu_bf16"]["device"] = torch.cuda.get_device_name(device)
+        LOG.info("GPU bf16 latency: %s", out["gpu_bf16"])
+    else:
+        out["gpu_bf16"] = None
+    model.cpu().float()
+    torch.set_num_threads(cpu_threads)
+    cpu = torch.device("cpu")
+    probe = _time_single(model, tok, texts[:3] + texts[:3], cpu, max_len, use_amp=False, warmup=3)
+    est_s = float(np.mean(probe)) / 1000.0 * n
+    n_cpu = n if est_s <= CPU_LATENCY_BUDGET_S or n <= CPU_LATENCY_FALLBACK_N else CPU_LATENCY_FALLBACK_N
+    if n_cpu != n:
+        LOG.info("CPU latency: %d samples would take ~%.0f s (> %.0f s budget); using %d", n, est_s, CPU_LATENCY_BUDGET_S, n_cpu)
+    cpu_times = _time_single(model, tok, texts[:3] + texts[:n_cpu], cpu, max_len, use_amp=False)
+    out["cpu_fp32"] = _summ(cpu_times, toks[:n_cpu])
+    out["cpu_fp32"]["threads"] = torch.get_num_threads()
+    out["cpu_fp32"]["estimated_full_run_s"] = round(est_s, 1)
+    LOG.info("CPU fp32 latency: %s", out["cpu_fp32"])
+    return out
+
+
+# ------------------------------------------------------------------------------ plots
+def _style(ax, title: str, xlabel: str, ylabel: str) -> None:
+    ax.set_title(title, color=C_INK, fontsize=11, loc="left")
+    ax.set_xlabel(xlabel, color=C_INK2)
+    ax.set_ylabel(ylabel, color=C_INK2)
+    ax.grid(True, color=C_GRID, linewidth=0.6)
+    for s in ("top", "right"):
+        ax.spines[s].set_visible(False)
+    for s in ("left", "bottom"):
+        ax.spines[s].set_color(C_GRID)
+    ax.tick_params(colors=C_INK2)
+    ax.set_facecolor(C_SURFACE)
+
+
+def _save(fig, path: Path) -> None:
+    fig.patch.set_facecolor(C_SURFACE)
+    fig.tight_layout()
+    fig.savefig(path, dpi=PLOT_DPI)
+    plt.close(fig)
+    LOG.info("wrote %s", path)
+
+
+def plot_confusion(cm: dict[str, int], path: Path, title: str) -> None:
+    mat = np.array([[cm["tn"], cm["fp"]], [cm["fn"], cm["tp"]]])
+    fig, ax = plt.subplots(figsize=(5.6, 4.8))
+    ConfusionMatrixDisplay(mat, display_labels=[BINARY_NAMES[0], BINARY_NAMES[1]]).plot(
+        ax=ax, cmap="Blues", values_format="d", colorbar=False)
+    ax.set_title(title, color=C_INK, fontsize=11, loc="left")
+    ax.set_xlabel("Predicted label", color=C_INK2)
+    ax.set_ylabel("True label", color=C_INK2)
+    ax.tick_params(colors=C_INK2)
+    plt.setp(ax.get_xticklabels(), rotation=15, ha="right", fontsize=8)
+    plt.setp(ax.get_yticklabels(), fontsize=8)
+    _save(fig, path)
+
+
+def plot_confusion3(mat: Sequence[Sequence[int]], path: Path, title: str) -> None:
+    fig, ax = plt.subplots(figsize=(5.6, 4.8))
+    ConfusionMatrixDisplay(np.asarray(mat), display_labels=[LABEL_NAMES[c] for c in range(3)]).plot(
+        ax=ax, cmap="Blues", values_format="d", colorbar=False)
+    ax.set_title(title, color=C_INK, fontsize=11, loc="left")
+    ax.set_xlabel("Predicted label (roll-up first, then argmax phishing/spam)", color=C_INK2, fontsize=9)
+    ax.set_ylabel("True label (corpus provenance)", color=C_INK2)
+    ax.tick_params(colors=C_INK2)
+    _save(fig, path)
+
+
+def plot_roc(tf: dict[str, Any], path: Path, title: str) -> None:
+    c = tf["curves"]
+    fig, axes = plt.subplots(1, 2, figsize=(10, 4.4))
+    for ax, logx in zip(axes, (False, True)):
+        ax.plot(c["fpr"], c["tpr"], color=C_BLUE, linewidth=2, label=f"ROC (AUC {tf['roc_auc']:.4f})")
+        if not logx:
+            ax.plot([0, 1], [0, 1], color=C_GRID, linewidth=1, linestyle="--")
+        for key, col, lab, dy in (("recall_at_fpr_1pct", C_ORANGE, "1% FPR", 16), ("recall_at_fpr_0_1pct", C_AQUA, "0.1% FPR", 4)):
+            r = tf[key]
+            x = max(r["fpr"], 1e-4 if logx else 0)
+            ax.scatter([x], [r["recall"]], color=col, s=48, zorder=3)
+            ax.annotate(f"recall {r['recall']:.3f} @ {lab}", (x, r["recall"]), textcoords="offset points",
+                        xytext=(8, dy), color=C_INK2, fontsize=8, annotation_clip=False)
+        if logx:
+            ax.set_xscale("log")
+            ax.set_xlim(1e-4, 1)
+            _style(ax, "ROC, log false-positive rate", "False positive rate (log)", "Recall (true positive rate)")
+        else:
+            ax.set_xlim(0, 1)
+            _style(ax, title, "False positive rate", "Recall (true positive rate)")
+        ax.set_ylim(0, 1.02)
+        ax.legend(loc="lower right", frameon=False, labelcolor=C_INK2)
+    _save(fig, path)
+
+
+def plot_pr(tf: dict[str, Any], base_rate: float, path: Path, title: str) -> None:
+    c = tf["curves"]
+    fig, ax = plt.subplots(figsize=(5.6, 4.4))
+    ax.plot(c["recall"], c["precision"], color=C_BLUE, linewidth=2, label=f"PR (AP {tf['pr_auc']:.4f})")
+    ax.axhline(base_rate, color=C_GRID, linewidth=1, linestyle="--")
+    ax.text(0.01, base_rate + 0.01, f"positive rate {base_rate:.3f}", color=C_INK2, fontsize=8)
+    b = tf["best_f1"]
+    ax.scatter([b["recall"]], [b["precision"]], color=C_ORANGE, s=48, zorder=3)
+    ax.annotate(f"best F1 {b['f1']:.3f} @ thr {b['threshold']:.3f}", (b["recall"], b["precision"]),
+                textcoords="offset points", xytext=(-8, -14), ha="right", color=C_INK2, fontsize=8)
+    ax.set_xlim(0, 1)
+    ax.set_ylim(0, 1.02)
+    _style(ax, title, "Recall (unwanted)", "Precision (unwanted)")
+    ax.legend(loc="lower left", frameon=False, labelcolor=C_INK2)
+    _save(fig, path)
+
+
+def _read_csv(path: Path):
+    import pandas as pd
+
+    try:
+        df = pd.read_csv(path)
+    except Exception as e:  # noqa: BLE001 - a corrupt CSV must not fail the evaluation
+        LOG.warning("could not read %s: %s", path, e)
+        return None
+    return df if len(df) else None
+
+
+def plot_pretrain_curves(run_dir: Path, path: Path) -> bool:
+    """Loss (train + val) and learning rate from the pretraining run's metrics.csv; two
+    panels, one axis each (never a twin axis)."""
+    df = _read_csv(run_dir / "metrics.csv")
+    if df is None or "step" not in df or "loss" not in df:
+        return False
+    fig, axes = plt.subplots(1, 2, figsize=(10, 4.2))
+    ax = axes[0]
+    ax.plot(df["step"], df["loss"], color=C_BLUE, linewidth=1.6, label="train MLM loss")
+    if "val_mlm_loss" in df and df["val_mlm_loss"].notna().any():
+        v = df[df["val_mlm_loss"].notna()]
+        ax.plot(v["step"], v["val_mlm_loss"], color=C_ORANGE, linewidth=2, marker="o", markersize=4, label="val MLM loss")
+    ax.axhline(math.log(32768), color=C_GRID, linewidth=1, linestyle="--")
+    ax.text(float(df["step"].min()), math.log(32768) + 0.05, "ln(V) = uniform guess", color=C_INK2, fontsize=8)
+    _style(ax, f"MLM pretraining loss ({run_dir.name})", "optimizer step", "cross-entropy")
+    ax.legend(frameon=False, labelcolor=C_INK2)
+    ax = axes[1]
+    if "lr" in df:
+        ax.plot(df["step"], df["lr"], color=C_AQUA, linewidth=2)
+        _style(ax, "Learning rate schedule", "optimizer step", "lr")
+        ax.ticklabel_format(axis="y", style="sci", scilimits=(0, 0))
+    else:
+        ax.axis("off")
+    _save(fig, path)
+    return True
+
+
+def plot_finetune_curves(run_dir: Path, path: Path) -> bool:
+    """Training loss per logging step (metrics.csv) and val metrics per evaluation
+    (epochs.csv). finetune.py appends an epochs.csv row every ``eval_every`` steps, not
+    only at epoch ends, so both panels use the optimizer ``step`` as the x-axis (plotting
+    against ``epoch`` stacks every mid-epoch evaluation on one vertical line); rows with
+    ``is_epoch_end == 1`` get a hollow marker."""
+    m = _read_csv(run_dir / "metrics.csv")
+    e = _read_csv(run_dir / "epochs.csv")
+    if m is None and e is None:
+        return False
+    fig, axes = plt.subplots(1, 2, figsize=(10, 4.2))
+    ax = axes[0]
+    if m is not None and "loss" in m:
+        x = m["step"] if "step" in m else np.arange(len(m))
+        ax.plot(x, m["loss"], color=C_BLUE, linewidth=1.4, label="train loss")
+        for col, colr in (("val_loss", C_ORANGE),):
+            if col in m and m[col].notna().any():
+                v = m[m[col].notna()]
+                ax.plot(v["step"] if "step" in v else v.index, v[col], color=colr, linewidth=2, marker="o", markersize=4, label=col)
+        _style(ax, f"Fine-tuning loss ({run_dir.name})", "optimizer step", "cross-entropy")
+        ax.legend(frameon=False, labelcolor=C_INK2)
+    else:
+        ax.axis("off")
+    ax = axes[1]
+    if e is not None:
+        x = e["step"] if "step" in e else (e["epoch"] if "epoch" in e else np.arange(1, len(e) + 1))
+        series = [c for c in ("f1", "recall", "precision", "accuracy", "roc_auc", "pr_auc", "macro_f1_3way") if c in e]
+        series = series or [c for c in e.columns if c.startswith("val_") and c != "val_loss"][:6]
+        colors = [C_BLUE, C_ORANGE, C_AQUA, "#eda100", "#e87ba4", "#008300", C_VIOLET]
+        for col, colr in zip(series, colors):
+            ax.plot(x, e[col], color=colr, linewidth=2, marker="o", markersize=4, label=col)
+            ax.annotate(f"{float(e[col].iloc[-1]):.3f}", (float(x.iloc[-1]), float(e[col].iloc[-1])),
+                        textcoords="offset points", xytext=(6, 0), color=C_INK2, fontsize=8)
+        if "is_epoch_end" in e and (e["is_epoch_end"] == 1).any():
+            ends = e[e["is_epoch_end"] == 1]
+            for xe in (ends["step"] if "step" in ends else ends.index):
+                ax.axvline(float(xe), color=C_GRID, linewidth=1, linestyle="--")
+            ax.text(float((ends["step"] if "step" in ends else ends.index).iloc[0]), 0.03, "epoch end", color=C_INK2,
+                    fontsize=8, rotation=90, va="bottom", ha="right", transform=ax.get_xaxis_transform())
+        _style(ax, "Validation metrics per evaluation", "optimizer step", "score")
+        if series:
+            ax.legend(frameon=False, labelcolor=C_INK2, fontsize=8)
+    else:
+        ax.axis("off")
+    _save(fig, path)
+    return True
+
+
+# --------------------------------------------------------------------- error analysis
+_WORD = re.compile(r"[a-z][a-z0-9'\-]{2,}")
+_HTML_TAG = re.compile(r"<\s*/?\s*[a-z!][^>]{0,80}>", re.I)
+_LIST_BODY = re.compile(r"unsubscribe|mailing list|listserv|majordomo|listinfo|to be removed from this list|digest", re.I)
+_LIST_SUBJ = re.compile(r"^\s*(re:\s*)?\[[^\]]{1,40}\]")
+
+
+def preview(subject: str, body: str, n: int = PREVIEW_CHARS) -> str:
+    """Subject and body on one line, whitespace collapsed, pipes escaped for the markdown table."""
+    s = re.sub(r"\s+", " ", f"{subject.strip()} │ {body.strip()}").replace("|", "\\|")
+    return (s[:n] + "...") if len(s) > n else s
+
+
+def is_mailing_list(subject: str, body: str) -> bool:
+    return bool(_LIST_SUBJ.search(subject) or _LIST_BODY.search(body))
+
+
+def is_html_heavy(body: str) -> bool:
+    return len(_HTML_TAG.findall(body)) >= 10 or bool(re.search(r"<\s*(html|table|td|tr)\b", body, re.I))
+
+
+def over_represented(a_texts: Sequence[str], b_texts: Sequence[str], top: int = 20, alpha: float = 0.5) -> list[dict[str, Any]]:
+    """Tokens with the highest document-frequency ratio in group A vs group B (smoothed).
+    Simple lexical statistics, not a model explanation: it says what the missed emails
+    tend to contain that caught ones do not."""
+    if not a_texts or not b_texts:
+        return []
+    df_a: Counter = Counter()
+    df_b: Counter = Counter()
+    for t in a_texts:
+        df_a.update(set(_WORD.findall(t.lower())))
+    for t in b_texts:
+        df_b.update(set(_WORD.findall(t.lower())))
+    min_df = max(3, int(0.05 * len(a_texts)))
+    rows = []
+    for w, c in df_a.items():
+        if c < min_df:
+            continue
+        ra, rb = (c + alpha) / (len(a_texts) + alpha), (df_b.get(w, 0) + alpha) / (len(b_texts) + alpha)
+        rows.append({"token": w, "share_a": round(c / len(a_texts), 3), "share_b": round(df_b.get(w, 0) / len(b_texts), 3),
+                     "ratio": round(ra / rb, 2)})
+    rows.sort(key=lambda r: (-r["ratio"], -r["share_a"]))
+    return rows[:top]
+
+
+def error_analysis(split: Split, enc: Encoded, probs: np.ndarray, p: np.ndarray, thr: float, at: dict[str, Any], tf: dict[str, Any],
+                   lure: dict[str, Any], conf: dict[str, Any], tw: ThreeWay, run_name: str, ts: str) -> tuple[str, dict[str, Any]]:
+    y = split.label
+    pred = (p >= thr).astype(np.int64)
+    fn_idx = np.nonzero((y == 1) & (pred == 0))[0]
+    fp_idx = np.nonzero((y == 0) & (pred == 1))[0]
+    tp_idx = np.nonzero((y == 1) & (pred == 1))[0]
+    tn_idx = np.nonzero((y == 0) & (pred == 0))[0]
+    fn_top = fn_idx[np.argsort(p[fn_idx])][:TOP_K_ERRORS]          # most confidently "legitimate"
+    fp_top = fp_idx[np.argsort(-p[fp_idx])][:TOP_K_ERRORS]         # most confidently "unwanted"
+
+    def share(idx: np.ndarray, fn) -> float | None:
+        return round(float(np.mean([fn(i) for i in idx])), 3) if len(idx) else None
+
+    comp = {
+        grp: {
+            "n": int(len(idx)),
+            "mailing_list_or_digest": share(idx, lambda i: is_mailing_list(split.subject[i], split.body[i])),
+            "html_heavy": share(idx, lambda i: is_html_heavy(split.body[i])),
+            "very_short_lt20_tokens": share(idx, lambda i: enc.n_tokens_full[i] - 2 < 20),
+            "truncated_gt512": share(idx, lambda i: bool(enc.truncated[i])),
+            "lure_phrase": share(idx, lambda i: bool(lure["mask"][i])),
+        }
+        for grp, idx in (("fp", fp_idx), ("tn", tn_idx), ("fn", fn_idx), ("tp", tp_idx))
+    }
+    fn_vs_tp = over_represented([split.text[i] for i in fn_idx], [split.text[i] for i in tp_idx])
+    fp_vs_tn = over_represented([split.text[i] for i in fp_idx], [split.text[i] for i in tn_idx])
+
+    cm = at["confusion_matrix"]
+    k = probs.shape[1]
+    L = [f"# Error analysis — {split.name} split ({SPLIT_TITLE[split.name]}), run `{run_name}` ({ts})", "",
+         "All quoted emails are rows of public corpora "
+         f"(source {', '.join(sorted(set(split.source)))}); nothing here is private mail. "
+         f"Threshold {thr}. Binary positive class = {UNWANTED_NAME}, P(unwanted) = "
+         + ("P(phishing) + P(spam)." if k == 3 else "P(class 1) of the 2-class head."), "",
+         "## Counts (binary roll-up)", "",
+         f"- rows {at['n']:,} (positives {at['n_positive']:,}, {100 * at['n_positive'] / max(at['n'], 1):.1f}%); "
+         f"label3 counts {lure_free_counts(split)}",
+         f"- **recall (unwanted) {at['recall']:.4f}** — {cm['fn']:,} missed unwanted emails (false negatives) out of {at['n_positive']:,}",
+         f"- precision {at['precision']:.4f} — {cm['fp']:,} false alarms out of {cm['fp'] + cm['tp']:,} flagged",
+         f"- F1 {at['f1']:.4f}, accuracy {at['accuracy']:.4f}, specificity {at['specificity']:.4f}",
+         f"- ROC-AUC {_fmt(tf['roc_auc'])}, PR-AUC {_fmt(tf['pr_auc'])}; "
+         f"recall at 1% FPR {_fmt(tf['recall_at_fpr_1pct'] and tf['recall_at_fpr_1pct']['recall'])}, "
+         f"at 0.1% FPR {_fmt(tf['recall_at_fpr_0_1pct'] and tf['recall_at_fpr_0_1pct']['recall'])}",
+         f"- credential-lure slice: {lure['metrics']['n']:,} rows, {lure['metrics']['n_positive']:,} positive "
+         f"({100 * lure['label_purity']:.1f}% label purity: slice precision cannot exceed what the phrase list selects), "
+         f"recall {_fmt(lure['metrics']['recall'])}, precision {_fmt(lure['metrics']['precision'])}, "
+         f"{lure['metrics']['confusion_matrix']['fn']} missed", ""]
+
+    def table(title: str, idx: np.ndarray) -> list[str]:
+        rows = [f"## {title}", "", "| # | doc_id | P(unwanted) | tokens (full) | source | label_set | preview (subject │ body, 160 chars) |",
+                "|---:|---|---:|---:|---|---|---|"]
+        for kk, i in enumerate(idx, 1):
+            rows.append(f"| {kk} | `{split.doc_id[i]}` | {p[i]:.4f} | {enc.n_tokens_full[i]} | {split.source[i]} | {split.label_set[i]} | "
+                        f"{preview(split.subject[i], split.body[i])} |")
+        if not len(idx):
+            rows.append("| - | none | | | | | |")
+        return rows + [""]
+
+    L += table(f"Top-{TOP_K_ERRORS} false negatives (missed unwanted mail, most confident first)", fn_top)
+    L += table(f"Top-{TOP_K_ERRORS} false positives (legitimate flagged, most confident first)", fp_top)
+
+    # three-way section
+    stats3: dict[str, Any] = {}
+    L += ["## Three-way (legitimate / phishing / spam)", ""]
+    if k != 3:
+        L += [f"Skipped: {tw.metrics.get('reason')}.", ""]
+    else:
+        m3 = tw.metrics
+        bd = m3["predicted_kind_breakdown"]
+        if tw.evaluated:
+            cmx = m3["confusion_matrix"]
+            L += [f"Decision rule: {DECISION_RULE_DOC}", "",
+                  f"Rows with a full label: {m3['rows_with_full_label']:,}; unwanted-but-unsplit rows (binary roll-up only): {m3['rows_unwanted_only']:,}.", "",
+                  "| true \\ predicted | " + " | ".join(cmx["labels"]) + " |", "|---|" + "---:|" * 3]
+            for i, lab in enumerate(cmx["labels"]):
+                L.append(f"| **{lab}** | " + " | ".join(f"{v:,}" for v in cmx["matrix"][i]) + " |")
+            L += ["", f"- accuracy {m3['accuracy']:.4f}, **macro-F1 {m3['macro_f1']:.4f}** (over 3 classes; over the "
+                  f"{len(m3['classes_present'])} present: {m3['macro_f1_present_classes']:.4f}); argmax-of-3 rule: macro-F1 "
+                  f"{m3['argmax_rule']['macro_f1']:.4f}, {m3['argmax_rule']['rows_where_rules_differ']:,} rows differ between the rules", "",
+                  "| class | precision | recall | F1 | support |", "|---|---:|---:|---:|---:|"]
+            for lab, v in m3["per_class"].items():
+                L.append(f"| {lab} | {v['precision']:.4f} | {v['recall']:.4f} | {v['f1']:.4f} | {v['support']:,} |")
+            pvs = m3["phish_vs_spam_on_true_unwanted"]
+            if pvs["kind_only"]:
+                ko, rr = pvs["kind_only"], pvs["rollup_rule"]
+                L += ["", f"### Phishing vs spam on the {pvs['n_true_unwanted']:,} true unwanted rows", "",
+                      f"Kind decision alone (given the row is unwanted, argmax(P(phishing), P(spam))): accuracy {ko['accuracy']:.4f}; "
+                      f"phishing read as spam {ko['phishing_as_spam']:,}, spam read as phishing {ko['spam_as_phishing']:,}.", "",
+                      "| true \\ predicted (roll-up rule) | Legitimate (missed) | Phishing | Spam |", "|---|---:|---:|---:|"]
+                for i, lab in enumerate(rr["rows"]):
+                    L.append(f"| **{lab}** | " + " | ".join(f"{v:,}" for v in rr["matrix"][i]) + " |")
+            L += ["", "### Per source (three-way rows)", "", "| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |",
+                  "|---|---:|---:|---:|---:|---:|---:|"]
+            for s, v in m3["per_source"].items():
+                c3 = v["counts"]
+                L.append(f"| {s} | {v['n']:,} | {c3['Legitimate']:,} | {c3['Phishing']:,} | {c3['Spam']:,} | {v['accuracy']:.4f} | {v['macro_f1_present_classes']:.4f} |")
+            # top confusions with previews
+            y3, pred3 = split.label3, tw.pred3
+            ps_idx = np.nonzero((y3 == 1) & (pred3 == 2))[0]
+            sp_idx = np.nonzero((y3 == 2) & (pred3 == 1))[0]
+            ps_top = ps_idx[np.argsort(-probs[ps_idx, 2])][:TOP_K_CONFUSIONS]
+            sp_top = sp_idx[np.argsort(-probs[sp_idx, 1])][:TOP_K_CONFUSIONS]
+
+            def table3(title: str, idx: np.ndarray) -> list[str]:
+                rows = [f"### {title}", "", "| # | doc_id | P(legit) | P(phish) | P(spam) | tokens (full) | source | preview (subject │ body, 160 chars) |",
+                        "|---:|---|---:|---:|---:|---:|---|---|"]
+                for kk, i in enumerate(idx, 1):
+                    rows.append(f"| {kk} | `{split.doc_id[i]}` | {probs[i, 0]:.4f} | {probs[i, 1]:.4f} | {probs[i, 2]:.4f} | {enc.n_tokens_full[i]} | "
+                                f"{split.source[i]} | {preview(split.subject[i], split.body[i])} |")
+                if not len(idx):
+                    rows.append("| - | none | | | | | | |")
+                return rows + [""]
+
+            L += [""]
+            L += table3(f"Top-{TOP_K_CONFUSIONS} phishing -> spam confusions ({len(ps_idx):,} total, most confident first)", ps_top)
+            L += table3(f"Top-{TOP_K_CONFUSIONS} spam -> phishing confusions ({len(sp_idx):,} total, most confident first)", sp_top)
+            ps_vs_pp = over_represented([split.text[i] for i in ps_idx], [split.text[i] for i in np.nonzero((y3 == 1) & (pred3 == 1))[0]])
+            sp_vs_ss = over_represented([split.text[i] for i in sp_idx], [split.text[i] for i in np.nonzero((y3 == 2) & (pred3 == 2))[0]])
+            L += ["### Tokens over-represented in phishing read as spam vs phishing read correctly", ""] + _ratio_table(ps_vs_pp, "phish->spam", "phish->phish")
+            L += ["", "### Tokens over-represented in spam read as phishing vs spam read correctly", ""] + _ratio_table(sp_vs_ss, "spam->phish", "spam->spam") + [""]
+            stats3 = {"n_phish_as_spam": int(len(ps_idx)), "n_spam_as_phish": int(len(sp_idx)),
+                      "top_phish_as_spam_doc_ids": [split.doc_id[i] for i in ps_top], "top_spam_as_phish_doc_ids": [split.doc_id[i] for i in sp_top],
+                      "phish_as_spam_tokens": ps_vs_pp, "spam_as_phish_tokens": sp_vs_ss}
+        else:
+            L += [f"No three-way ground truth on this split: {m3['reason']}.", ""]
+        tpf, fpf, mis = bd["true_positives_flagged"], bd["false_positives_flagged"], bd["missed_positives"]
+        L += ["### Predicted kind of the flagged rows (informational — "
+              + ("no phishing-vs-spam ground truth on this split)" if not tw.evaluated else "includes the unsplit rows, which have no kind label)"), "",
+              "| group | n | predicted Phishing | predicted Spam |", "|---|---:|---:|---:|",
+              f"| true positives flagged | {tpf['n']:,} | {tpf['predicted_phishing']:,} | {tpf['predicted_spam']:,} |",
+              f"| false positives flagged | {fpf['n']:,} | {fpf['predicted_phishing']:,} | {fpf['predicted_spam']:,} |",
+              f"| missed positives (kind the model would have given) | {mis['n']:,} | {mis['predicted_phishing']:,} | {mis['predicted_spam']:,} |", ""]
+
+    L += ["## Patterns (simple lexical statistics, not a model explanation)", "",
+          "### Tokens over-represented in false negatives vs true positives (document frequency, smoothed ratio)", ""]
+    L += _ratio_table(fn_vs_tp, "FN", "TP")
+    L += ["", "### Tokens over-represented in false positives vs true negatives", ""]
+    L += _ratio_table(fp_vs_tn, "FP", "TN")
+    L += ["", "### Composition of the error groups (share of rows)", "",
+          "| group | n | mailing-list / digest | HTML-heavy | very short (<20 content tokens) | truncated (>512) | lure phrase |",
+          "|---|---:|---:|---:|---:|---:|---:|"]
+    for grp in ("fp", "tn", "fn", "tp"):
+        c = comp[grp]
+        L.append(f"| {grp.upper()} | {c['n']:,} | {_pct(c['mailing_list_or_digest'])} | {_pct(c['html_heavy'])} | "
+                 f"{_pct(c['very_short_lt20_tokens'])} | {_pct(c['truncated_gt512'])} | {_pct(c['lure_phrase'])} |")
+    L += ["", "### Length confound (RUN_LOG entry 2)", "",
+          "| group | n | mean tokens (full) | mean tokens fed | median (full) | % truncated |", "|---|---:|---:|---:|---:|---:|"]
+    for grp in ("tp", "tn", "fp", "fn", "correct", "wrong"):
+        s = conf["by_outcome"][grp]
+        L.append(f"| {grp.upper()} | {s['n']:,} | {_fmt(s['mean_tokens_full'], 1)} | {_fmt(s['mean_tokens_fed'], 1)} | "
+                 f"{_fmt(s['median_tokens_full'], 0)} | {_fmt(s['pct_truncated'], 1)} |")
+    for lab in ("legitimate", "unwanted"):
+        s = conf["by_label"][lab]
+        L.append(f"| all {lab} | {s['n']:,} | {_fmt(s['mean_tokens_full'], 1)} | {_fmt(s['mean_tokens_fed'], 1)} | "
+                 f"{_fmt(s['median_tokens_full'], 0)} | {_fmt(s['pct_truncated'], 1)} |")
+    for lab, s in conf["by_class3"].items():
+        if lab in ("phish", "spam", "unwanted"):
+            L.append(f"| all label3={lab} | {s['n']:,} | {_fmt(s['mean_tokens_full'], 1)} | {_fmt(s['mean_tokens_fed'], 1)} | "
+                     f"{_fmt(s['median_tokens_full'], 0)} | {_fmt(s['pct_truncated'], 1)} |")
+    L += ["", "| length bucket (full tokens) | n | positive rate | recall | FPR | accuracy |", "|---|---:|---:|---:|---:|---:|"]
+    for b in conf["by_length_bucket"]:
+        L.append(f"| {b['bucket']} | {b['n']:,} | {b['positive_rate']:.3f} | {_fmt(b['recall'])} | {_fmt(b['fpr'])} | {b['accuracy']:.4f} |")
+
+    L += ["", "## Summary", ""] + summary_sentences(at, tf, lure, conf, comp, fn_vs_tp, fp_vs_tn, tw)
+    stats = {"n_fn": int(len(fn_idx)), "n_fp": int(len(fp_idx)), "composition": comp,
+             "fn_vs_tp_tokens": fn_vs_tp, "fp_vs_tn_tokens": fp_vs_tn,
+             "top_fn_doc_ids": [split.doc_id[i] for i in fn_top], "top_fp_doc_ids": [split.doc_id[i] for i in fp_top],
+             "threeway": stats3 or None}
+    return "\n".join(L) + "\n", stats
+
+
+def lure_free_counts(split: Split) -> str:
+    c = _label3_counts(split.label3)
+    return ", ".join(f"{k} {v:,}" for k, v in c.items() if v)
+
+
+def summary_sentences(at, tf, lure, conf, comp, fn_vs_tp, fp_vs_tn, tw: ThreeWay | None = None) -> list[str]:
+    cm = at["confusion_matrix"]
+    out = [f"- The classifier catches **{100 * at['recall']:.2f}% of unwanted mail (phishing or spam)** at threshold {at['threshold']} "
+           f"({cm['fn']:,} of {at['n_positive']:,} missed) while flagging {100 * at['false_positive_rate']:.2f}% of legitimate mail "
+           f"({cm['fp']:,} false alarms). Recall is the headline because a missed phish is the costly error for a small-business inbox."]
+    if tf["recall_at_fpr_1pct"]:
+        r1 = tf["recall_at_fpr_1pct"]
+        thr_txt = f"threshold {r1['threshold']:.3f}, flag when p >= threshold" if r1["threshold"] is not None else "no finite threshold reaches this budget"
+        out.append(f"- Holding false alarms to 1 in 100 legitimate emails keeps recall at {100 * r1['recall']:.2f}% "
+                   f"({thr_txt}); at 1 in 1,000 it is "
+                   f"{100 * tf['recall_at_fpr_0_1pct']['recall']:.2f}%. The best-F1 threshold is {tf['best_f1']['threshold']:.3f} "
+                   f"(F1 {tf['best_f1']['f1']:.4f}).")
+    if tw is not None and tw.evaluated:
+        m3 = tw.metrics
+        pc = m3["per_class"]
+        pvs = m3["phish_vs_spam_on_true_unwanted"]
+        s = (f"- Three-way: macro-F1 **{m3['macro_f1']:.4f}** over {m3['rows_with_full_label']:,} rows with a full label "
+             f"(recall legitimate {100 * pc['Legitimate']['recall']:.1f}%, phishing {100 * pc['Phishing']['recall']:.1f}%, spam {100 * pc['Spam']['recall']:.1f}%).")
+        if pvs["kind_only"]:
+            ko = pvs["kind_only"]
+            s += (f" Given that an email is unwanted, the model tells phishing from spam {100 * ko['accuracy']:.1f}% of the time "
+                  f"({ko['phishing_as_spam']:,} phishing read as spam, {ko['spam_as_phishing']:,} spam read as phishing). "
+                  "The phishing/spam labels follow corpus provenance (Nazario + Nigerian fraud + synthetic lures = phishing; "
+                  "SpamAssassin + Enron-Spam spam folders = spam), so part of this confusion is corpus style, not lure content.")
+        out.append(s)
+    elif tw is not None and tw.metrics.get("num_labels") == 3:
+        bd = tw.metrics["predicted_kind_breakdown"]["true_positives_flagged"]
+        out.append(f"- Three-way: this split has no phishing-vs-spam ground truth; of the {bd['n']:,} true positives the model flagged, "
+                   f"it called {bd['predicted_phishing']:,} phishing and {bd['predicted_spam']:,} spam (informational only).")
+    lm = lure["metrics"]
+    if lm["n_positive"] >= LURE_MIN_POSITIVES:
+        out.append(f"- On the {lm['n']:,}-row credential-lure slice ({len(CREDENTIAL_LURE_PHRASES)} documented phrases, "
+                   f"{100 * lure['label_purity']:.1f}% of its rows are labelled positive) recall is "
+                   f"{100 * lm['recall']:.2f}% with precision {100 * lm['precision']:.2f}%: "
+                   f"{'the highest-cost phish are caught at least as well as the average' if lm['recall'] >= at['recall'] else 'the highest-cost phish are caught LESS often than the average positive, which is the first thing to fix'}.")
+    elif lm["n"]:
+        out.append(f"- The credential-lure slice has only {lm['n']:,} rows ({lm['n_positive']:,} positive, {lm['confusion_matrix']['fn']} missed): "
+                   f"too few rows to compare its recall with the overall recall (needs >= {LURE_MIN_POSITIVES} positives).")
+    else:
+        out.append("- No row of this sample contains a credential-lure phrase; the slice is empty.")
+    bo, bl = conf["by_outcome"], conf["by_label"]
+    mean = {g: bo[g]["mean_tokens_full"] for g in ("tp", "tn", "fp", "fn")}
+    out.append(f"- Length confound: missed unwanted mail averages {_fmt(mean['fn'], 0)} tokens vs {_fmt(mean['tp'], 0)} for caught; "
+               f"false positives average {_fmt(mean['fp'], 0)} tokens vs {_fmt(mean['tn'], 0)} for correct legitimate mail "
+               f"(all legitimate {_fmt(bl['legitimate']['mean_tokens_full'], 0)}, all unwanted {_fmt(bl['unwanted']['mean_tokens_full'], 0)}).")
+    if all(mean[g] is not None for g in mean):
+        follows_length = mean["fn"] > mean["tp"] and mean["fp"] < mean["tn"]
+        out.append(f"  Legitimate mail is ~{bl['legitimate']['mean_tokens_full'] / max(bl['unwanted']['mean_tokens_full'] or 1, 1):.1f}x longer than unwanted mail on average; "
+                   + ("the errors sit on long unwanted mail and short legitimate mail, so the model partly reads length rather than content."
+                      if follows_length else "the error groups do not follow the length gap, so length is not the main shortcut."))
+    c3 = conf.get("by_class3") or {}
+    if "phish" in c3 and "spam" in c3:
+        out.append(f"  Phishing rows average {_fmt(c3['phish']['mean_tokens_full'], 0)} tokens vs spam {_fmt(c3['spam']['mean_tokens_full'], 0)}: "
+                   "a length gap between the two unwanted kinds is a second shortcut to watch in the phishing-vs-spam confusion.")
+    buckets = [b for b in conf["by_length_bucket"] if b["recall"] is not None and b["n_positive"] >= 20]
+    if buckets:
+        worst = min(buckets, key=lambda b: b["recall"])
+        out.append(f"- Recall by length is lowest in the {worst['bucket']}-token bucket ({100 * worst['recall']:.1f}%, {worst['n_positive']:,} positives).")
+    fp_c = comp["fp"]
+    if fp_c["n"]:
+        out.append(f"- Of the {fp_c['n']:,} false positives, {_pct(fp_c['mailing_list_or_digest'])} look like mailing-list or digest traffic, "
+                   f"{_pct(fp_c['html_heavy'])} are HTML-heavy and {_pct(fp_c['very_short_lt20_tokens'])} are very short "
+                   f"(TN reference: {_pct(comp['tn']['mailing_list_or_digest'])} / {_pct(comp['tn']['html_heavy'])} / {_pct(comp['tn']['very_short_lt20_tokens'])}).")
+    if fn_vs_tp:
+        out.append("- Tokens most over-represented in missed unwanted mail vs caught: " + ", ".join(f"`{r['token']}`" for r in fn_vs_tp[:10]) + ".")
+    if fp_vs_tn:
+        out.append("- Tokens most over-represented in false positives vs correct legitimate mail: " + ", ".join(f"`{r['token']}`" for r in fp_vs_tn[:10]) + ".")
+    return out
+
+
+def _ratio_table(rows: list[dict[str, Any]], a: str, b: str) -> list[str]:
+    if not rows:
+        return ["(too few rows in one of the groups)"]
+    out = [f"| token | share of {a} docs | share of {b} docs | ratio |", "|---|---:|---:|---:|"]
+    out += [f"| `{r['token']}` | {r['share_a']:.3f} | {r['share_b']:.3f} | {r['ratio']:.2f} |" for r in rows]
+    return out
+
+
+def _fmt(v: Any, nd: int = 4) -> str:
+    if v is None or v is False:
+        return "n/a"
+    return f"{v:.{nd}f}"
+
+
+def _pct(v: float | None) -> str:
+    return "n/a" if v is None else f"{100 * v:.1f}%"
+
+
+# ------------------------------------------------------------------------ run log
+def next_run_log_number(path: Path) -> int:
+    if not path.exists():
+        return 0
+    nums = [int(m.group(1)) for m in re.finditer(r"^## (\d+)\.", path.read_text(encoding="utf-8"), re.M)]
+    return (max(nums) + 1) if nums else 0
+
+
+@dataclass
+class SplitResult:
+    """Everything one split produced, kept until the invocation-level artifacts are written."""
+
+    split: Split
+    enc: Encoded
+    probs: np.ndarray
+    p_unw: np.ndarray
+    inf: dict[str, Any]
+    at: dict[str, Any]
+    tf: dict[str, Any]
+    lure: dict[str, Any]
+    per_source: dict[str, Any]
+    conf: dict[str, Any]
+    tw: ThreeWay
+    plots: dict[str, str]
+    err_path: Path
+    err_stats: dict[str, Any]
+
+
+def _split_table(r: SplitResult) -> list[str]:
+    at, tf, lure, tw = r.at, r.tf, r.lure, r.tw
+    cm = at["confusion_matrix"]
+    lines = [f"### {r.split.name} split ({SPLIT_TITLE[r.split.name]}): `{r.split.path.name}`, {r.split.n:,} rows "
+             f"({at['n_positive']:,} unwanted, {100 * at['n_positive'] / max(r.split.n, 1):.1f}%; label3 {lure_free_counts(r.split)}). "
+             f"Inference {r.inf['rows_per_s']} rows/s on {r.inf['device']} (bf16 autocast, batch {r.inf['batch_size']}, length-bucketed), "
+             f"peak VRAM {r.inf['peak_vram_gb']} GB.", "",
+             "| binary roll-up metric (P(unwanted) = P(phishing) + P(spam)) | value |", "|---|---:|",
+             f"| **recall (unwanted)** | **{at['recall']:.4f}** ({cm['fn']:,} missed of {at['n_positive']:,}) |",
+             f"| precision | {at['precision']:.4f} ({cm['fp']:,} false alarms) |",
+             f"| F1 | {at['f1']:.4f} |", f"| accuracy | {at['accuracy']:.4f} |", f"| specificity | {at['specificity']:.4f} |",
+             f"| ROC-AUC | {_fmt(tf['roc_auc'])} |", f"| PR-AUC | {_fmt(tf['pr_auc'])} |",
+             f"| recall @ 1% FPR | {_fmt(tf['recall_at_fpr_1pct'] and tf['recall_at_fpr_1pct']['recall'])} |",
+             f"| recall @ 0.1% FPR | {_fmt(tf['recall_at_fpr_0_1pct'] and tf['recall_at_fpr_0_1pct']['recall'])} |",
+             f"| best-F1 threshold | {_fmt(tf['best_f1'] and tf['best_f1']['threshold'], 3)} (F1 {_fmt(tf['best_f1'] and tf['best_f1']['f1'])}) |",
+             f"| credential-lure slice | {lure['metrics']['n']:,} rows ({100 * lure['label_purity']:.1f}% labelled positive), "
+             f"recall {_fmt(lure['metrics']['recall'])}, precision {_fmt(lure['metrics']['precision'])} |", ""]
+    if tw.evaluated:
+        m3 = tw.metrics
+        cmx = m3["confusion_matrix"]
+        pc = m3["per_class"]
+        lines += [f"**Three-way** on the {m3['rows_with_full_label']:,} rows with label3 in {{0,1,2}} ({m3['rows_unwanted_only']:,} unsplit rows "
+                  f"enter the roll-up only; rule: {DECISION_RULE}): accuracy {m3['accuracy']:.4f}, **macro-F1 {m3['macro_f1']:.4f}**; "
+                  + "; ".join(f"{lab} P {v['precision']:.3f} / R {v['recall']:.3f} / F1 {v['f1']:.3f} (n={v['support']:,})" for lab, v in pc.items())
+                  + ". 3x3 (rows true, cols predicted Legitimate/Phishing/Spam): "
+                  + "; ".join(f"{lab} {'/'.join(f'{v:,}' for v in cmx['matrix'][i])}" for i, lab in enumerate(cmx["labels"])) + "."]
+        pvs = m3["phish_vs_spam_on_true_unwanted"]
+        if pvs["kind_only"]:
+            ko = pvs["kind_only"]
+            lines[-1] += (f" Phishing-vs-spam on the {pvs['n_true_unwanted']:,} true unwanted rows: kind accuracy {ko['accuracy']:.4f} "
+                          f"({ko['phishing_as_spam']:,} phishing->spam, {ko['spam_as_phishing']:,} spam->phishing).")
+        lines += ["", "| source | n | Legitimate | Phishing | Spam | accuracy | macro-F1 (present classes) |", "|---|---:|---:|---:|---:|---:|---:|"]
+        for s, v in m3["per_source"].items():
+            c3 = v["counts"]
+            lines.append(f"| {s} | {v['n']:,} | {c3['Legitimate']:,} | {c3['Phishing']:,} | {c3['Spam']:,} | {v['accuracy']:.4f} | {v['macro_f1_present_classes']:.4f} |")
+        lines.append("")
+    elif tw.metrics.get("num_labels") == 3:
+        bd = tw.metrics["predicted_kind_breakdown"]["true_positives_flagged"]
+        lines += [f"**Three-way:** no phishing-vs-spam ground truth on this split ({tw.metrics['reason']}); informational: of the "
+                  f"{bd['n']:,} flagged true positives the model called {bd['predicted_phishing']:,} phishing and {bd['predicted_spam']:,} spam.", ""]
+    else:
+        lines += [f"**Three-way:** skipped ({tw.metrics.get('reason')}).", ""]
+    return lines
+
+
+def run_log_entry(num: int, ts: str, run_name: str, results: list[SplitResult], ckpt: Path, lat: dict, out_dir: Path, dev: bool, k: int) -> str:
+    g, c = lat.get("gpu_bf16"), lat.get("cpu_fp32")
+    names = " + ".join(r.split.name for r in results)
+    lines = [f"## {num}. {ts} — Evaluation on the {names} split{'s' if len(results) > 1 else ''} (evaluate.py run, `{run_name}`)"
+             f"{' [DEV RUN, --limit-rows]' if dev else ''}", "",
+             f"**Checkpoint.** `{ckpt}` ({k}-class head{'; three-way label contract of RUN_LOG entry 6' if k == 3 else ''}). "
+             "Binary roll-up metrics use the same definitions as the binary-era runs so they stay comparable.", ""]
+    for r in results:
+        lines += _split_table(r)
+    lines += ["**Latency (batch 1, tokenize + forward + softmax, timeit.default_timer) vs the instructor's 74 ms.** "
+              + (f"GPU bf16: {g['avg_ms']} ms avg / {g['p50_ms']} ms p50 over {g['n']} emails ({g['ratio_to_benchmark']:.2f}x). " if g else "GPU: not available. ")
+              + (f"CPU fp32 ({c['threads']} threads): {c['avg_ms']} ms avg / {c['p50_ms']} ms p50 over {c['n']} emails ({c['ratio_to_benchmark']:.2f}x)." if c else ""),
+              ""]
+    r0 = results[0]
+    conf = r0.conf
+    lines += [f"**Length confound ({r0.split.name}).** FN mean {_fmt(conf['by_outcome']['fn']['mean_tokens_full'], 0)} tokens vs TP {_fmt(conf['by_outcome']['tp']['mean_tokens_full'], 0)}; "
+              f"FP {_fmt(conf['by_outcome']['fp']['mean_tokens_full'], 0)} vs TN {_fmt(conf['by_outcome']['tn']['mean_tokens_full'], 0)} "
+              f"(see error_analysis{r0.split.sfx}.md for the per-bucket table).", "",
+              f"**Artifacts.** `{out_dir}`: " + "; ".join(
+                  f"metrics{r.split.sfx}.json, latency{r.split.sfx}.json, confusion_matrix{r.split.sfx}.png"
+                  + (f", confusion_matrix3{r.split.sfx}.png" if r.tw.evaluated else "")
+                  + f", roc_curve{r.split.sfx}.png, pr_curve{r.split.sfx}.png, error_analysis{r.split.sfx}.md" for r in results)
+              + " (training curves: reports/pretrain_curves.png and finetune_curves.png from pretrain.py / finetune.py `plot`)."]
+    return "\n".join(lines)
+
+
+# -------------------------------------------------------------------------------- main
+def _summary(r: SplitResult) -> dict[str, Any]:
+    """The experiments.jsonl summary for one split: the binary keys of the binary-era line
+    plus a ``threeway`` sub-dict (None when not evaluated)."""
+    at, tf, lure, tw = r.at, r.tf, r.lure, r.tw
+    s = {"accuracy": at["accuracy"], "precision": at["precision"], "recall": at["recall"], "f1": at["f1"],
+         "specificity": at["specificity"], "roc_auc": tf["roc_auc"], "pr_auc": tf["pr_auc"],
+         "recall_at_fpr_1pct": tf["recall_at_fpr_1pct"] and tf["recall_at_fpr_1pct"]["recall"],
+         "recall_at_fpr_0_1pct": tf["recall_at_fpr_0_1pct"] and tf["recall_at_fpr_0_1pct"]["recall"],
+         "confusion_matrix": at["confusion_matrix"], "n": r.split.n, "threshold": at["threshold"],
+         "lure_recall": lure["metrics"]["recall"] if lure["metrics"]["n_positive"] else None,
+         "f1_unwanted": at["f1"], "threeway": None}
+    if tw.evaluated:
+        m3 = tw.metrics
+        pvs = m3["phish_vs_spam_on_true_unwanted"]
+        s["threeway"] = {"n": m3["rows_with_full_label"], "accuracy": m3["accuracy"], "macro_f1": m3["macro_f1"],
+                         "macro_f1_present_classes": m3["macro_f1_present_classes"],
+                         "per_class": {lab: {"precision": v["precision"], "recall": v["recall"], "f1": v["f1"], "support": v["support"]}
+                                       for lab, v in m3["per_class"].items()},
+                         "confusion_matrix_3x3": m3["confusion_matrix"]["matrix"],
+                         "kind_accuracy_on_true_unwanted": pvs["kind_only"]["accuracy"] if pvs["kind_only"] else None}
+        s["macro_f1_3way"] = m3["macro_f1"]
+    return s
+
+
+def evaluate_split(name: str, model, mcfg: EncoderConfig, tok, cfg: dict, args: argparse.Namespace, device: torch.device,
+                   max_len: int, out_dir: Path, run_name: str, ts: str, data_dir: Path) -> SplitResult:
+    dev = args.limit_rows is not None
+    test3_file = str((cfg.get("eval") or {}).get("test3_file") or SPLIT_FILES["test3"])
+    split = load_split(data_dir, name, args.limit_rows, int(cfg["seed"]), test3_file)
+    LOG.info("%s split: %d rows, %d unwanted (%.1f%%), label3 %s, sources %s%s", split.name, split.n, int(split.label.sum()),
+             100 * split.label.mean(), _label3_counts(split.label3), sorted(set(split.source)), f" [limit_rows={args.limit_rows}]" if dev else "")
+    t0 = common.timer()
+    enc = encode_split(tok, split.text, max_len)
+    LOG.info("encoded %d rows in %.1fs: mean %.0f tokens (full), %.1f%% truncated at %d",
+             split.n, common.timer() - t0, enc.n_tokens_full.mean(), 100 * enc.truncated.mean(), max_len)
+
+    probs, inf = predict_probs(model, enc, device, args.batch_size, int(cfg["finetune"].get("dynamic_padding_multiple", 8)))
+    p_unw = p_unwanted(probs)
+    thr = args.threshold
+    at = threshold_metrics(split.label, p_unw, thr)
+    tf = threshold_free_metrics(split.label, p_unw)
+    lmask, per_phrase, phrase_by_label = lure_mask(split.text, split.label)
+    lure = {"mask": lmask, "phrases": list(CREDENTIAL_LURE_PHRASES), "phrase_hits": dict(per_phrase.most_common()),
+            "phrase_hits_by_label": phrase_by_label,
+            "metrics": slice_metrics(split.label[lmask], p_unw[lmask], thr),
+            "label3_counts": _label3_counts(split.label3[lmask])}
+    # Share of slice rows labelled positive: the slice is defined by phrases, not labels, so its
+    # "precision" is bounded by this purity and must be read next to it.
+    lure["label_purity"] = _safe_div(lure["metrics"]["n_positive"], lure["metrics"]["n"])
+    per_source = per_source_table(split, p_unw, thr)
+    conf = length_confound(split, enc, p_unw, thr)
+    tw = threeway_metrics(split, probs, p_unw, thr)
+    if tw.evaluated and lmask.any():
+        full = lmask & (split.label3 >= 0)
+        lure["threeway_accuracy"] = float((tw.pred3[full] == split.label3[full]).mean()) if full.any() else None
+    if not tw.evaluated:
+        LOG.info("three-way sections skipped on %s: %s", split.name, tw.metrics.get("reason"))
+
+    cm = at["confusion_matrix"]
+    print(f"\n=== {split.name} split / {SPLIT_TITLE[split.name]} ({split.n:,} rows, {at['n_positive']:,} unwanted) — run {run_name} ===")
+    print(f"RECALL (unwanted = phishing or spam, the metric that matters): {at['recall']:.4f}   missed: {cm['fn']:,} / {at['n_positive']:,}")
+    print(f"precision {at['precision']:.4f}  F1 {at['f1']:.4f}  accuracy {at['accuracy']:.4f}  specificity {at['specificity']:.4f}")
+    print(f"confusion matrix [[TN {cm['tn']:,}, FP {cm['fp']:,}], [FN {cm['fn']:,}, TP {cm['tp']:,}]]")
+    print(f"ROC-AUC {_fmt(tf['roc_auc'])}  PR-AUC {_fmt(tf['pr_auc'])}  "
+          f"recall@1%FPR {_fmt(tf['recall_at_fpr_1pct'] and tf['recall_at_fpr_1pct']['recall'])}  "
+          f"recall@0.1%FPR {_fmt(tf['recall_at_fpr_0_1pct'] and tf['recall_at_fpr_0_1pct']['recall'])}  "
+          f"best-F1 thr {_fmt(tf['best_f1'] and tf['best_f1']['threshold'], 3)}")
+    lm = lure["metrics"]
+    print(f"credential-lure slice: {lm['n']:,} rows ({lm['n_positive']:,} positive = {100 * lure['label_purity']:.1f}% label purity)  "
+          f"recall {_fmt(lm['recall'])}  precision {_fmt(lm['precision'])}"
+          + (f"  [< {LURE_MIN_POSITIVES} positives: not compared to overall recall]" if lm['n_positive'] < LURE_MIN_POSITIVES else ""))
+    print("per source: " + "; ".join(f"{s}: n={m['n']:,} recall={_fmt(m['recall'])} prec={_fmt(m['precision'])}" for s, m in per_source.items()))
+    bo = conf["by_outcome"]
+    print(f"mean tokens (full): TP {_fmt(bo['tp']['mean_tokens_full'], 0)}  FN {_fmt(bo['fn']['mean_tokens_full'], 0)}  "
+          f"TN {_fmt(bo['tn']['mean_tokens_full'], 0)}  FP {_fmt(bo['fp']['mean_tokens_full'], 0)}")
+    if tw.evaluated:
+        m3 = tw.metrics
+        print(f"THREE-WAY ({m3['rows_with_full_label']:,} rows with label3 in {{0,1,2}}): accuracy {m3['accuracy']:.4f}  MACRO-F1 {m3['macro_f1']:.4f}  "
+              + "  ".join(f"{lab[:5]} P/R/F1 {v['precision']:.3f}/{v['recall']:.3f}/{v['f1']:.3f} (n={v['support']:,})" for lab, v in m3["per_class"].items()))
+        print("3x3 [rows true L/P/S, cols pred L/P/S]: " + str(m3["confusion_matrix"]["matrix"]))
+        pvs = m3["phish_vs_spam_on_true_unwanted"]
+        if pvs["kind_only"]:
+            ko = pvs["kind_only"]
+            print(f"phishing vs spam on {pvs['n_true_unwanted']:,} true unwanted rows: kind accuracy {ko['accuracy']:.4f}  "
+                  f"phish->spam {ko['phishing_as_spam']:,}  spam->phish {ko['spam_as_phishing']:,}")
+        print("per source (3-way): " + "; ".join(f"{s}: n={v['n']:,} L/P/S={v['counts']['Legitimate']}/{v['counts']['Phishing']}/{v['counts']['Spam']} "
+                                                  f"acc={v['accuracy']:.3f}" for s, v in m3["per_source"].items()))
+    elif mcfg.num_labels == 3:
+        bd = tw.metrics["predicted_kind_breakdown"]["true_positives_flagged"]
+        print(f"three-way: no ground truth on this split ({tw.metrics['reason']}); flagged true positives split "
+              f"{bd['predicted_phishing']:,} phishing / {bd['predicted_spam']:,} spam (informational)")
+    else:
+        print(f"three-way: skipped ({tw.metrics.get('reason')})")
+
+    sfx = split.sfx
+    plots: dict[str, str] = {}
+    p_cm = out_dir / f"confusion_matrix{sfx}.png"
+    plot_confusion(cm, p_cm, f"Confusion matrix (binary roll-up) — {split.name} split, threshold {thr}")
+    plots["confusion_matrix"] = str(p_cm)
+    if tw.evaluated:
+        p_cm3 = out_dir / f"confusion_matrix3{sfx}.png"
+        plot_confusion3(tw.metrics["confusion_matrix"]["matrix"], p_cm3, f"Three-way confusion matrix — {split.name} split, threshold {thr}")
+        plots["confusion_matrix3"] = str(p_cm3)
+    if tf["curves"] is not None:
+        p_roc, p_pr = out_dir / f"roc_curve{sfx}.png", out_dir / f"pr_curve{sfx}.png"
+        plot_roc(tf, p_roc, f"ROC (binary roll-up) — {split.name} split")
+        plot_pr(tf, split.label.mean(), p_pr, f"Precision-recall (binary roll-up) — {split.name} split")
+        plots["roc_curve"], plots["pr_curve"] = str(p_roc), str(p_pr)
+    # Training curves are pretrain.py / finetune.py `plot` outputs (reports/pretrain_curves.png,
+    # reports/finetune_curves.png) and are never regenerated here: evaluate.py cannot know which
+    # pretraining run a scratch-initialised fine-tune "belongs" to, and a wrong guess would silently
+    # overwrite the authoritative plot. Explicit run dirs opt into extra eval_* renderings.
+    for flag, plotter, key in ((args.finetune_run_dir, plot_finetune_curves, "finetune"), (args.pretrain_run_dir, plot_pretrain_curves, "pretrain")):
+        if not flag:
+            continue
+        p_curve = out_dir / f"eval_{key}_curves{sfx}.png"
+        if plotter(Path(flag), p_curve):
+            plots[f"eval_{key}_curves"] = str(p_curve)
+        else:
+            LOG.warning("no usable metrics.csv under %s: %s skipped", flag, p_curve.name)
+
+    md, err_stats = error_analysis(split, enc, probs, p_unw, thr, at, tf, lure, conf, tw, run_name, ts)
+    p_err = out_dir / f"error_analysis{sfx}.md"
+    p_err.write_text(md, encoding="utf-8")
+    LOG.info("wrote %s", p_err)
+    return SplitResult(split, enc, probs, p_unw, inf, at, tf, lure, per_source, conf, tw, plots, p_err, err_stats)
+
+
+def run(args: argparse.Namespace) -> int:
+    cfg = common.load_config(args.config)
+    global LOG, TOP_K_CONFUSIONS
+    LOG = common.setup_logging("evaluate", cfg["paths"]["logs"])
+    TOP_K_CONFUSIONS = check_eval_config(cfg)
+    common.set_seed(int(cfg["seed"]))
+    common.print_device_info()
+    device = torch.device("cuda" if torch.cuda.is_available() and not args.cpu else "cpu")
+    ts = datetime.now().strftime("%Y-%m-%d %H:%M")
+    dev = args.limit_rows is not None
+    splits = [s.strip() for s in args.split.split(",") if s.strip()]
+    bad = [s for s in splits if s not in SPLIT_FILES]
+    if bad or not splits:
+        raise ValueError(f"--split must be a comma list of {sorted(SPLIT_FILES)}, got {args.split!r}")
+    if len(set(splits)) != len(splits):
+        raise ValueError(f"--split lists a split twice: {args.split!r}")
+
+    ckpt = Path(args.checkpoint).resolve()
+    run_name = args.run_name or (ckpt.parent.name if ckpt.name in ("best", "last") else ckpt.name)
+    out_dir = Path(args.out_dir) if args.out_dir else Path(cfg["paths"]["reports"])
+    out_dir.mkdir(parents=True, exist_ok=True)
+    data_dir = Path(args.data_dir) if args.data_dir else Path(cfg["paths"]["data_processed"])
+    LOG.info("run_name=%s splits=%s checkpoint=%s out_dir=%s data_dir=%s device=%s", run_name, splits, ckpt, out_dir, data_dir, device)
+
+    model, mcfg, ckpt_cfg = load_classifier(ckpt, device)
+    k = int(mcfg.num_labels)
+    max_len = int(args.max_len or mcfg.max_seq_len)
+    tok = load_tokenizer(cfg["paths"]["tokenizer"])
+    if tok.get_vocab_size() > mcfg.vocab_size:
+        raise RuntimeError(f"tokenizer vocab {tok.get_vocab_size()} > model vocab {mcfg.vocab_size}")
+    if k == 2:
+        LOG.info("2-class checkpoint: binary roll-up only; every three-way section is skipped")
+    else:
+        LOG.info("3-class checkpoint: labels %s; binary roll-up P(unwanted) = P(phishing) + P(spam); rule %s", LABEL_NAMES, DECISION_RULE)
+
+    # Inference + per-split artifacts first (the model stays on `device`); latency last (it
+    # moves the model to the CPU), measured once per invocation on the first split's sample.
+    results = [evaluate_split(name, model, mcfg, tok, cfg, args, device, max_len, out_dir, run_name, ts, data_dir) for name in splits]
+    r0 = results[0]
+    lat = measure_latency(model, tok, r0.split, r0.enc, device, max_len, args.latency_samples, args.cpu_threads, int(cfg["seed"]))
+    lat.update({"checkpoint": str(ckpt), "run_name": run_name, "params": count_parameters(model)["total"], "timestamp": ts,
+                "num_labels": k, "measured_once_for_splits": splits})
+    g, c = lat.get("gpu_bf16"), lat["cpu_fp32"]
+    print(f"\nlatency per email (batch 1, tokenize+forward+softmax) vs instructor benchmark {INSTRUCTOR_BENCHMARK_MS:.0f} ms:")
+    if g:
+        print(f"  GPU bf16 : avg {g['avg_ms']:.2f} ms  p50 {g['p50_ms']:.2f} ms  (n={g['n']}, {g['ratio_to_benchmark']:.2f}x benchmark)")
+    print(f"  CPU fp32 : avg {c['avg_ms']:.1f} ms  p50 {c['p50_ms']:.1f} ms  (n={c['n']}, {c['threads']} threads, {c['ratio_to_benchmark']:.2f}x benchmark)")
+
+    for r in results:
+        split, sfx = r.split, r.split.sfx
+        common.write_json(lat, out_dir / f"latency{sfx}.json")
+        tf_out = {kk: v for kk, v in r.tf.items() if kk != "curves"}
+        metrics = {
+            "timestamp": ts, "run_name": run_name, "split": split.name, "split_file": str(split.path), "split_title": SPLIT_TITLE[split.name],
+            "limit_rows": args.limit_rows, "dev_run": dev, "checkpoint": str(ckpt), "splits_in_invocation": splits,
+            "model_config": mcfg.to_dict(), "num_labels": k, "params": count_parameters(model)["total"],
+            "checkpoint_config": {kk: v for kk, v in ckpt_cfg.items() if kk != "model"},
+            "tokenizer": str(cfg["paths"]["tokenizer"]), "max_len": max_len,
+            "label_names": {str(c3): LABEL_NAMES[c3] for c3 in range(k)} if k == 3 else {str(c2): BINARY_NAMES[c2] for c2 in range(2)},
+            "binary_rollup": {"unwanted_name": UNWANTED_NAME, "classes": list(range(1, k)), "rule": "P(unwanted) = sum of P(class) for class >= 1; flag when >= threshold"},
+            "split_sizes": {"rows": split.n, "positive": r.at["n_positive"], "negative": r.at["n_negative"],
+                            "positive_rate": round(_safe_div(r.at["n_positive"], split.n), 4),
+                            "label3": _label3_counts(split.label3),
+                            "by_source": {s: {"rows": m["n"], "positive": m["n_positive"], "label3": m["label3_counts"]} for s, m in r.per_source.items()},
+                            "mean_tokens_full": round(float(r.enc.n_tokens_full.mean()), 1),
+                            "pct_truncated": round(100 * float(r.enc.truncated.mean()), 2)},
+            "threshold": args.threshold, "at_threshold": r.at, "threshold_free": tf_out,
+            "threeway": r.tw.metrics,
+            "credential_lure_slice": {"n_phrases": len(CREDENTIAL_LURE_PHRASES), "phrases": r.lure["phrases"],
+                                      "match_rule": "whole-word match on NFKC-lowercased text with non-alphanumerics collapsed to spaces",
+                                      "phrase_hits": r.lure["phrase_hits"], "phrase_hits_by_label": r.lure["phrase_hits_by_label"],
+                                      "label_purity": round(r.lure["label_purity"], 4), "min_positives_for_comparison": LURE_MIN_POSITIVES,
+                                      "label3_counts": r.lure["label3_counts"], "threeway_accuracy": r.lure.get("threeway_accuracy"),
+                                      "metrics": r.lure["metrics"]},
+            "per_source": r.per_source, "length_confound": r.conf, "inference": r.inf, "latency": lat,
+            "error_analysis": {"path": str(r.err_path), **r.err_stats}, "plots": r.plots,
+        }
+        p_metrics = out_dir / f"metrics{sfx}.json"
+        common.write_json(metrics, p_metrics)
+        LOG.info("wrote %s", p_metrics)
+
+    # experiments.jsonl (contract: one line per invocation with run_name + the metrics of every split run)
+    by_name = {r.split.name: _summary(r) for r in results}
+    line = {"run_name": run_name, "timestamp": ts, "stage": "evaluate", "checkpoint": str(ckpt), "splits": splits, "num_labels": k,
+            "dev_run": dev, "limit_rows": args.limit_rows,
+            "val": by_name.get("val"), "test": by_name.get("test"), "test3": by_name.get("test3"),
+            "latency_ms": {"gpu_bf16_avg": g and g["avg_ms"], "cpu_fp32_avg": c["avg_ms"]}}
+    p_exp = out_dir / "experiments.jsonl"
+    with p_exp.open("a", encoding="utf-8") as fh:
+        fh.write(json.dumps(line, default=common._json_default) + "\n")
+    LOG.info("appended %s", p_exp)
+
+    # RUN_LOG.md: one entry per invocation covering every split that was run
+    num = next_run_log_number(common.project_root() / "RUN_LOG.md")
+    entry = run_log_entry(num, ts, run_name, results, ckpt, lat, out_dir, dev, k)
+    p_entry = out_dir / f"run_log_entry_{'_'.join(splits)}.md"   # test -> _test, test,test3 -> _test_test3 (no collision with test3)
+    p_entry.write_text(entry + "\n", encoding="utf-8")
+    if dev or args.no_run_log:
+        LOG.info("RUN_LOG.md not appended (%s); entry saved to %s", "dev run" if dev else "--no-run-log", p_entry)
+    else:
+        common.append_run_log(entry)
+        LOG.info("appended entry %d to RUN_LOG.md", num)
+    return 0
+
+
+def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--config", default="config.yaml")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    r = sub.add_parser("run", help="evaluate a fine-tuned checkpoint on the test / test3 / val split(s)")
+    r.add_argument("--checkpoint", required=True, help="fine-tune best/ dir with model.safetensors + config.json")
+    r.add_argument("--split", default="test",
+                   help=f"one of {sorted(SPLIT_FILES)} or a comma list (e.g. test,test3): one invocation, one experiments line, one RUN_LOG entry")
+    r.add_argument("--limit-rows", type=int, default=None, help="dev only: seeded random subset per split; skips the RUN_LOG append")
+    r.add_argument("--batch-size", type=int, default=64)
+    r.add_argument("--latency-samples", type=int, default=None, help="default config.eval.latency_samples (100)")
+    r.add_argument("--out-dir", default=None, help="default config.paths.reports")
+    r.add_argument("--data-dir", default=None, help="default config.paths.data_processed (fixture dir for self-tests)")
+    r.add_argument("--run-name", default=None, help="default: the run dir name (parent of best/)")
+    r.add_argument("--threshold", type=float, default=DEFAULT_THRESHOLD)
+    r.add_argument("--max-len", type=int, default=None, help="default model max_seq_len")
+    r.add_argument("--cpu-threads", type=int, default=16)
+    r.add_argument("--cpu", action="store_true", help="force CPU inference (dev)")
+    r.add_argument("--finetune-run-dir", default=None,
+                   help="opt-in: also render <out-dir>/eval_finetune_curves.png from this run's metrics.csv/epochs.csv "
+                        "(reports/finetune_curves.png itself belongs to finetune.py plot)")
+    r.add_argument("--pretrain-run-dir", default=None,
+                   help="opt-in: also render <out-dir>/eval_pretrain_curves.png from this run's metrics.csv "
+                        "(reports/pretrain_curves.png itself belongs to pretrain.py plot)")
+    r.add_argument("--no-run-log", action="store_true", help="do not append to RUN_LOG.md")
+    args = ap.parse_args(argv)
+    if args.latency_samples is None:
+        args.latency_samples = int(common.load_config(args.config)["eval"]["latency_samples"])
+    return args
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    args = parse_args(argv)
+    return run(args)
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
diff --git a/presentation/src/export.py b/presentation/src/export.py
new file mode 100644
index 0000000000000000000000000000000000000000..1762a6a2f18b78f953b79e0a6500b51a61766690
--- /dev/null
+++ b/presentation/src/export.py
@@ -0,0 +1,758 @@
+"""Export a fine-tuned classifier for the Hugging Face model repo and the Gradio Space.
+
+Subcommand ``run --checkpoint <finetune best dir>``:
+
+1. load ``EncoderForClassification`` strictly (``num_labels`` from ``config.json["model"]``),
+   cast to ``--dtype`` (bf16 by default: ~337M params x 2 B = ~675 MB, half the fp32 size,
+   and the Space runs the encoder under bf16 anyway; the classifier has no tied weights,
+   so a plain ``state_dict`` is safetensors-safe) and write ``model.safetensors`` with a
+   text-only metadata header (format, dtype, architecture, config JSON) so a loader can
+   rebuild the model from the file alone. The file is re-read and checked key-for-key.
+2. ``tokenizer.json`` (copied) and ``config.json`` = model config + label names
+   (``{"0": "Legitimate", "1": "Phishing", "2": "Spam"}`` for a 3-class head, binary names
+   otherwise) + ``binary_rollup`` (which class indices sum to "unwanted") + max_seq_len +
+   export dtype + source checkpoint + the evaluation metrics (reports/metrics.json for the
+   TREC-06 binary table and reports/metrics_test3.json for the three-way table, when
+   present) + the fine-tune dials and init descriptor + the pretraining summary (the
+   pretrain run's final/config.json reached through the checkpoint's ``pretrained:<path>``
+   descriptor or ``--pretrain-config``; always null for a scratch-initialised checkpoint,
+   never guessed from the newest run).
+3. ``README.md`` model card (huggingface_hub ModelCard / ModelCardData front matter) with
+   BOTH metric tables, stating prominently that no pretrained weights were used and that
+   the phishing / spam labels follow corpus provenance (Nazario + Nigerian fraud + synthetic
+   lures = phishing; SpamAssassin + Enron-Spam spam folders = spam; TREC / CEAS positives
+   were trained as "unwanted" only).
+4. copy ``src/model.py`` -> ``<app-dir>/model.py`` byte-for-byte (sha1 asserted) and the
+   three weight files into ``<app-dir>/`` so ``app.py`` can load locally when they exist
+   and fall back to ``hf_hub_download`` otherwise. ``app.py`` and ``requirements.txt`` belong
+   to another owner and are never written here.
+
+Nothing is uploaded; ``upload_to_hf.py`` owns that step.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import logging
+import shutil
+import sys
+from datetime import datetime
+from pathlib import Path
+from typing import Any, Sequence
+
+sys.path.insert(0, str(Path(__file__).parent))
+
+import torch  # noqa: E402
+from safetensors import safe_open  # noqa: E402
+from safetensors.torch import load_file, save_file  # noqa: E402
+
+import common  # noqa: E402
+from model import EncoderConfig, EncoderForClassification, count_parameters  # noqa: E402
+
+LOG = logging.getLogger("export")
+
+# Label contract (RUN_LOG entry 6): 0 legitimate, 1 phishing, 2 spam; the binary roll-up "unwanted" is
+# the sum over every non-legitimate class. A 2-class head's class 1 IS the roll-up, hence its name.
+UNWANTED_NAME = "Unwanted (phishing or spam)"
+LABELS_3: dict[str, str] = {"0": "Legitimate", "1": "Phishing", "2": "Spam"}
+LABELS_2: dict[str, str] = {"0": "Legitimate", "1": UNWANTED_NAME}
+LABEL_PROVENANCE = ("Phishing = Nazario phishing corpus + Nigerian-fraud (advance-fee) corpus + synthetic BITS-style credential/BEC lures; "
+                    "Spam = SpamAssassin and Enron-Spam spam folders; the TREC-05 / TREC-07 / CEAS-08 positives never separated spam "
+                    "from phishing and were trained as 'unwanted' only (probability mass of phishing + spam), never as either class.")
+DTYPES: dict[str, torch.dtype] = {"bf16": torch.bfloat16, "fp32": torch.float32}
+WEIGHT_FILES: tuple[str, ...] = ("model.safetensors", "tokenizer.json", "config.json")
+PROTECTED_APP_FILES: frozenset[str] = frozenset({"app.py", "requirements.txt"})
+DATASET_IDS: tuple[str, str] = ("puyang2025/seven-phishing-email-datasets", "corbt/enron-emails")
+AURA_DATASET_ID = "kudzaiprichard/aura-phishing-email-corpus"   # Champa et al. mirror: Nazario.csv + Nigerian_Fraud.csv (phishing)
+CARD_TAGS: tuple[str, ...] = ("phishing", "email-security", "text-classification", "from-scratch")
+
+
+def labels_for(num_labels: int) -> dict[str, str]:
+    if num_labels == 3:
+        return dict(LABELS_3)
+    if num_labels == 2:
+        return dict(LABELS_2)
+    raise ValueError(f"num_labels={num_labels}: expected 2 or 3")
+
+
+def binary_rollup_for(num_labels: int) -> dict[str, list[int]]:
+    """Class indices whose probabilities sum to P(unwanted): [1, 2] for the 3-class head, [1] for binary."""
+    return {"unwanted": list(range(1, num_labels))}
+
+
+# ------------------------------------------------------------------------------ load
+def load_classifier(ckpt_dir: Path) -> tuple[EncoderForClassification, EncoderConfig, dict]:
+    cfg_json = common.read_json(ckpt_dir / "config.json")
+    mcfg = EncoderConfig.from_dict(cfg_json["model"])
+    mcfg.grad_checkpointing = False
+    labels_for(mcfg.num_labels)   # raises on an unsupported head size before anything is written
+    model = EncoderForClassification(mcfg)
+    sd = load_file(str(ckpt_dir / "model.safetensors"), device="cpu")
+    model.load_state_dict(sd, strict=True)
+    model.eval()
+    LOG.info("loaded %s: %d tensors, %s params, num_labels=%d", ckpt_dir, len(sd), f"{count_parameters(model)['total']:,}", mcfg.num_labels)
+    return model, mcfg, cfg_json
+
+
+def finetune_init(ckpt_cfg: dict) -> str | None:
+    """finetune.py's init descriptor as stored in the checkpoint config: ``"scratch"`` or
+    ``"pretrained:<path>"`` (top-level ``init`` or ``finetune.init``); None when absent."""
+    v = ckpt_cfg.get("init") or (ckpt_cfg.get("finetune") or {}).get("init")
+    return str(v) if v else None
+
+
+def is_scratch_init(ckpt_cfg: dict) -> bool:
+    return (finetune_init(ckpt_cfg) or "").strip().lower() == "scratch"
+
+
+def find_pretrain_summary(ckpt_cfg: dict, explicit: str | None) -> tuple[dict | None, str | None]:
+    """``training`` block of the pretraining run's final/config.json. Provenance has to be
+    exact, so only two sources are accepted: an explicit ``--pretrain-config`` path, or a
+    ``pretrained:<path>`` descriptor written by finetune.py into the checkpoint config.
+    There is deliberately no "newest run" fallback: a classifier fine-tuned from random
+    init (``init: scratch``) has no pretraining run, and a pretrained-init checkpoint
+    whose path no longer resolves gets ``None`` (the card then says so) rather than
+    another run's step counts. Passing ``--pretrain-config`` for a scratch checkpoint is
+    an error, not a hint."""
+    if is_scratch_init(ckpt_cfg):
+        if explicit:
+            raise ValueError(f"--pretrain-config given but the checkpoint was fine-tuned from scratch (init={finetune_init(ckpt_cfg)!r})")
+        return None, None
+    cands: list[Path] = []
+    if explicit:
+        cands.append(Path(explicit))
+
+    def walk(o: Any) -> None:
+        if isinstance(o, dict):
+            for v in o.values():
+                walk(v)
+        elif isinstance(o, list):
+            for v in o:
+                walk(v)
+        elif isinstance(o, str) and o.startswith("pretrained:"):
+            p = Path(o[len("pretrained:"):])
+            cands.append(p if p.is_absolute() else common.project_root() / p)
+
+    walk(ckpt_cfg)
+    for c in cands:
+        for p in (c, c / "config.json", c.parent / "config.json", c / "final" / "config.json"):
+            if p.is_file() and p.name == "config.json":
+                try:
+                    j = common.read_json(p)
+                except (OSError, json.JSONDecodeError):
+                    continue
+                if "training" in j:
+                    return j["training"], str(p)
+    return None, None
+
+
+# ----------------------------------------------------------------------------- export
+def export_weights(model: EncoderForClassification, mcfg: EncoderConfig, dtype: str, path: Path) -> dict[str, Any]:
+    model.to(DTYPES[dtype])
+    sd = {k: v.contiguous() for k, v in model.state_dict().items()}
+    meta = {"format": "pt", "dtype": dtype, "architecture": "EncoderForClassification",
+            "config": json.dumps(mcfg.to_dict(), sort_keys=True), "labels": json.dumps(labels_for(mcfg.num_labels)),
+            "exported": datetime.now().isoformat(timespec="seconds")}
+    path.parent.mkdir(parents=True, exist_ok=True)
+    save_file(sd, str(path), metadata=meta)
+    # Verify by re-reading: same keys, same shapes, requested dtype, metadata intact.
+    back = load_file(str(path), device="cpu")
+    if back.keys() != sd.keys():
+        raise RuntimeError("exported safetensors keys differ from the state_dict")
+    bad = [k for k in sd if back[k].shape != sd[k].shape or back[k].dtype != DTYPES[dtype]]
+    if bad:
+        raise RuntimeError(f"exported tensors mismatch: {bad[:5]}")
+    head = back["classifier.out.weight"].shape[0]
+    if head != mcfg.num_labels:
+        raise RuntimeError(f"classifier head has {head} outputs but config num_labels={mcfg.num_labels}")
+    with safe_open(str(path), framework="pt") as f:
+        if f.metadata().get("architecture") != "EncoderForClassification":
+            raise RuntimeError("safetensors metadata missing")
+    size = path.stat().st_size
+    LOG.info("wrote %s: %d tensors, %s, %.1f MB", path, len(sd), dtype, size / 1e6)
+    return {"tensors": len(sd), "bytes": size, "dtype": dtype, "params": count_parameters(model)["total"]}
+
+
+def build_config(mcfg: EncoderConfig, ckpt: Path, ckpt_cfg: dict, dtype: str, metrics: dict | None, metrics_path: str | None,
+                 metrics3: dict | None, metrics3_path: str | None, training: dict | None, training_path: str | None,
+                 metrics_mismatch: Sequence[str] = (), metrics3_mismatch: Sequence[str] = ()) -> dict[str, Any]:
+    scratch = is_scratch_init(ckpt_cfg)
+    m3s = _metrics3_summary(metrics3) if metrics3 else None   # None when the file has no evaluated three-way block
+    if scratch and training is not None:
+        raise AssertionError("scratch-initialised checkpoint must not carry a pretraining summary")
+    finetune = {k: v for k, v in ckpt_cfg.items() if k != "model"}
+    k = mcfg.num_labels
+    return {
+        "architecture": "EncoderForClassification",
+        "model": mcfg.to_dict(),
+        "num_labels": k,
+        "labels": labels_for(k),
+        "binary_rollup": binary_rollup_for(k),
+        "unwanted_name": UNWANTED_NAME,
+        "label_provenance": LABEL_PROVENANCE if k == 3 else "class 1 = unwanted mail (phishing or spam; the public corpora do not separate them)",
+        "max_seq_len": mcfg.max_seq_len,
+        "export_dtype": dtype,
+        "source_checkpoint": str(ckpt),
+        "exported": datetime.now().isoformat(timespec="seconds"),
+        "tokenizer": "tokenizer.json",
+        "model_input": 'subject + "\\n\\n" + body',
+        "finetune_checkpoint": finetune or None,
+        "finetune_init": finetune_init(ckpt_cfg),
+        "mlm_pretraining": "none (classifier trained from random init)" if scratch else ("summary below" if training else "yes, summary unavailable at export time"),
+        "metrics": _metrics_summary(metrics) if metrics else None,
+        "metrics_source": metrics_path,
+        "metrics_checkpoint_matches": (not metrics_mismatch) if metrics else None,
+        "metrics_mismatch": list(metrics_mismatch) or None,
+        "metrics_threeway": m3s,
+        # source only when a table is carried; otherwise say why (2-class head, or a split without phishing/spam ground truth)
+        "metrics_threeway_source": metrics3_path if m3s is not None else None,
+        "metrics_threeway_reason": None if m3s is not None else (((metrics3 or {}).get("threeway") or {}).get("reason")
+                                                                  or ("2-class checkpoint: no phishing-vs-spam head" if k == 2 else None)),
+        "metrics_threeway_checkpoint_matches": (not metrics3_mismatch) if m3s is not None else None,
+        "metrics_threeway_mismatch": list(metrics3_mismatch) or None,
+        "training": training,
+        "training_source": training_path,
+        "from_scratch": True,
+        "external_pretrained_weights_used": "none",
+    }
+
+
+def read_splits_manifest(data_processed: Path) -> dict | None:
+    """data_prep's splits_manifest.json, the single source for the dataset row counts quoted
+    in the model card (never hard-coded, so a re-run of data_prep cannot stale the card)."""
+    p = data_processed / "splits_manifest.json"
+    if not p.is_file():
+        return None
+    try:
+        return common.read_json(p)
+    except (OSError, json.JSONDecodeError) as e:
+        LOG.warning("could not read %s: %s", p, e)
+        return None
+
+
+def _metrics_summary(m: dict) -> dict[str, Any]:
+    """Binary roll-up summary of an evaluate.py metrics.json (the TREC-06 table)."""
+    at, tf, lat = m.get("at_threshold", {}), m.get("threshold_free", {}), m.get("latency", {})
+    r1, r01 = tf.get("recall_at_fpr_1pct") or {}, tf.get("recall_at_fpr_0_1pct") or {}
+    lure = (m.get("credential_lure_slice") or {}).get("metrics") or {}
+    g, c = lat.get("gpu_bf16") or {}, lat.get("cpu_fp32") or {}
+    tw = m.get("threeway") or {}
+    bd = (tw.get("predicted_kind_breakdown") or {}).get("true_positives_flagged")
+    return {
+        "split": m.get("split"), "split_file": m.get("split_file"), "n_rows": (m.get("split_sizes") or {}).get("rows"),
+        "dev_run": m.get("dev_run"), "threshold": m.get("threshold"), "timestamp": m.get("timestamp"), "num_labels": m.get("num_labels"),
+        "positive_class": UNWANTED_NAME,
+        "accuracy": at.get("accuracy"), "precision": at.get("precision"), "recall": at.get("recall"), "f1": at.get("f1"),
+        "specificity": at.get("specificity"), "confusion_matrix": at.get("confusion_matrix"),
+        "roc_auc": tf.get("roc_auc"), "pr_auc": tf.get("pr_auc"),
+        "recall_at_fpr_1pct": r1.get("recall"), "recall_at_fpr_0_1pct": r01.get("recall"),
+        "best_f1_threshold": (tf.get("best_f1") or {}).get("threshold"),
+        "credential_lure_recall": lure.get("recall"), "credential_lure_precision": lure.get("precision"), "credential_lure_n": lure.get("n"),
+        "predicted_kind_of_flagged_true_positives": bd,   # informational: no phishing-vs-spam ground truth on TREC-06
+        "latency_ms": {"gpu_bf16_avg": g.get("avg_ms"), "gpu_bf16_p50": g.get("p50_ms"), "cpu_fp32_avg": c.get("avg_ms"),
+                       "cpu_fp32_p50": c.get("p50_ms"), "cpu_threads": c.get("threads"),
+                       "instructor_benchmark_ms": lat.get("instructor_benchmark_ms")},
+    }
+
+
+def _metrics3_summary(m: dict) -> dict[str, Any] | None:
+    """Three-way summary of an evaluate.py metrics_test3.json; None when the file carries no
+    evaluated three-way block (2-class checkpoint, or a split without label3 ground truth)."""
+    tw = m.get("threeway") or {}
+    if not tw.get("evaluated"):
+        return None
+    at = m.get("at_threshold", {})
+    pvs = tw.get("phish_vs_spam_on_true_unwanted") or {}
+    ko = pvs.get("kind_only") or {}
+    return {
+        "split": m.get("split"), "split_file": m.get("split_file"), "n_rows": (m.get("split_sizes") or {}).get("rows"),
+        "dev_run": m.get("dev_run"), "threshold": m.get("threshold"), "timestamp": m.get("timestamp"),
+        "rows_with_full_label": tw.get("rows_with_full_label"), "label3_counts": tw.get("label3_counts"),
+        "decision_rule": tw.get("decision_rule"), "accuracy": tw.get("accuracy"), "macro_f1": tw.get("macro_f1"),
+        "per_class": tw.get("per_class"), "confusion_matrix": tw.get("confusion_matrix"),
+        "kind_accuracy_on_true_unwanted": ko.get("accuracy"), "phishing_as_spam": ko.get("phishing_as_spam"), "spam_as_phishing": ko.get("spam_as_phishing"),
+        "n_true_unwanted": pvs.get("n_true_unwanted"),
+        "per_source": {s: {"n": v.get("n"), "counts": v.get("counts"), "accuracy": v.get("accuracy")} for s, v in (tw.get("per_source") or {}).items()},
+        "binary_rollup_on_this_split": {"recall": at.get("recall"), "precision": at.get("precision"), "f1": at.get("f1"), "accuracy": at.get("accuracy")},
+    }
+
+
+# --------------------------------------------------------------------------- model card
+def _f(v: Any, nd: int = 4) -> str:
+    return "n/a" if v is None else f"{v:.{nd}f}"
+
+
+def _n(v: Any) -> str:
+    """Thousands-separated integer or 'n/a' (manifest fields may be missing)."""
+    return "n/a" if v is None else f"{int(v):,}"
+
+
+def _ms(v: Any) -> str:
+    """Latency cell: 'n/a' when that device was not measured (e.g. a CPU-only evaluation)."""
+    return "n/a" if v is None else f"{v} ms"
+
+
+def _binary_table_md(ms: dict[str, Any], metrics: dict) -> str:
+    cm = ms.get("confusion_matrix") or {}
+    lat = ms.get("latency_ms") or {}
+    lines = [
+        f"Evaluated by `src/evaluate.py` on the **{ms['split']}** split (`{Path(ms['split_file'] or '').name}`, {ms['n_rows']:,} rows, "
+        f"threshold {ms['threshold']}){' — DEV RUN on a row subset, not the final number' if ms.get('dev_run') else ''}. "
+        f"Positive class = **{UNWANTED_NAME}**"
+        + (", P(unwanted) = P(phishing) + P(spam); these definitions are identical to the binary-era runs, so the numbers are comparable." if ms.get("num_labels") == 3 else "."), "",
+        "| metric | value |", "|---|---:|",
+        f"| **Recall (unwanted)** | **{_f(ms['recall'])}** ({cm.get('fn', 'n/a')} missed of {(cm.get('fn') or 0) + (cm.get('tp') or 0):,}) |",
+        f"| Precision (unwanted) | {_f(ms['precision'])} |", f"| F1 (unwanted) | {_f(ms['f1'])} |",
+        f"| Accuracy | {_f(ms['accuracy'])} |", f"| Specificity | {_f(ms['specificity'])} |",
+        f"| ROC-AUC | {_f(ms['roc_auc'])} |", f"| PR-AUC | {_f(ms['pr_auc'])} |",
+        f"| Recall at 1% FPR | {_f(ms['recall_at_fpr_1pct'])} |", f"| Recall at 0.1% FPR | {_f(ms['recall_at_fpr_0_1pct'])} |",
+        f"| Credential-lure slice recall / precision (n={ms['credential_lure_n']}) | {_f(ms['credential_lure_recall'])} / {_f(ms['credential_lure_precision'])} |",
+        f"| Latency, batch 1, GPU bf16 (avg / p50) | {_ms(lat.get('gpu_bf16_avg'))} / {_ms(lat.get('gpu_bf16_p50'))} |",
+        f"| Latency, batch 1, CPU fp32, {lat.get('cpu_threads') or 'n/a'} threads (avg / p50) | {_ms(lat.get('cpu_fp32_avg'))} / {_ms(lat.get('cpu_fp32_p50'))} |",
+        f"| Instructor benchmark | {lat.get('instructor_benchmark_ms')} ms per sample |", "",
+        "Recall on the unwanted class is the headline number: in a defensive triage aid a missed phish is the "
+        "costly error, while a false positive costs a human one glance.",
+    ]
+    bd = ms.get("predicted_kind_of_flagged_true_positives")
+    if bd:
+        lines += ["", f"TREC-06 carries no phishing-vs-spam ground truth, so the three-way head can only be *described* here: of the "
+                  f"{bd['n']:,} true positives it flagged, the model called {bd['predicted_phishing']:,} phishing and {bd['predicted_spam']:,} spam "
+                  "(informational, not a metric)."]
+    return "\n".join(lines)
+
+
+def _threeway_table_md(m3: dict[str, Any]) -> str:
+    pc = m3.get("per_class") or {}
+    cmx = m3.get("confusion_matrix") or {}
+    labs = cmx.get("labels") or list(pc)
+    lines = [
+        f"Evaluated by `src/evaluate.py --split {m3['split']}` on `{Path(m3['split_file'] or '').name}` ({_n(m3['rows_with_full_label'])} rows with a "
+        f"full three-way label, threshold {m3['threshold']}){' — DEV RUN on a row subset, not the final number' if m3.get('dev_run') else ''}. "
+        f"Decision rule `{m3['decision_rule']}`: a row is unwanted iff P(phishing) + P(spam) >= threshold, then the kind is "
+        "argmax(P(phishing), P(spam)), so this table collapses exactly onto the binary roll-up.", "",
+        "| metric | value |", "|---|---:|",
+        f"| **Macro-F1 (3 classes)** | **{_f(m3['macro_f1'])}** |", f"| Accuracy | {_f(m3['accuracy'])} |",
+    ]
+    for lab, v in pc.items():
+        lines.append(f"| {lab}: precision / recall / F1 (n={v.get('support', 0):,}) | {_f(v.get('precision'))} / {_f(v.get('recall'))} / {_f(v.get('f1'))} |")
+    if m3.get("kind_accuracy_on_true_unwanted") is not None:
+        lines.append(f"| Phishing-vs-spam accuracy on the {_n(m3['n_true_unwanted'])} true unwanted rows (kind decision alone) | "
+                     f"{_f(m3['kind_accuracy_on_true_unwanted'])} ({_n(m3['phishing_as_spam'])} phishing read as spam, {_n(m3['spam_as_phishing'])} spam read as phishing) |")
+    br = m3.get("binary_rollup_on_this_split") or {}
+    lines.append(f"| Binary roll-up on this split: recall / precision / F1 | {_f(br.get('recall'))} / {_f(br.get('precision'))} / {_f(br.get('f1'))} |")
+    if cmx.get("matrix"):
+        lines += ["", "Confusion matrix (rows = true label by corpus provenance, columns = predicted):", "",
+                  "| true \\ predicted | " + " | ".join(labs) + " |", "|---|" + "---:|" * len(labs)]
+        for i, lab in enumerate(labs):
+            lines.append(f"| **{lab}** | " + " | ".join(f"{v:,}" for v in cmx["matrix"][i]) + " |")
+    ps = m3.get("per_source") or {}
+    if ps:
+        lines += ["", "| source | rows | Legitimate | Phishing | Spam | accuracy |", "|---|---:|---:|---:|---:|---:|"]
+        for s, v in ps.items():
+            c = v.get("counts") or {}
+            lines.append(f"| {s} | {_n(v.get('n'))} | {_n(c.get('Legitimate'))} | {_n(c.get('Phishing'))} | {_n(c.get('Spam'))} | {_f(v.get('accuracy'))} |")
+    lines += ["", f"**Label provenance.** {LABEL_PROVENANCE} The three-way holdout is drawn from text the tokenizer never saw and the pretraining "
+              "corpus does not contain (Nazario / Nigerian-fraud phishing, SpamAssassin / Enron-Spam spam, CEAS-08 / TREC-07 / SpamAssassin legitimate "
+              "mail from the former validation split). Because the kind labels come from which corpus a message sits in, part of any phishing-vs-spam "
+              "confusion measures corpus style rather than lure content."]
+    return "\n".join(lines)
+
+
+def model_card(mcfg: EncoderConfig, cfg: dict, ckpt_cfg: dict, metrics: dict | None, metrics3: dict | None, training: dict | None, dtype: str,
+               params: int, weight_bytes: int, model_repo: str, manifest: dict | None = None,
+               metrics_mismatch: Sequence[str] = (), metrics3_mismatch: Sequence[str] = ()) -> str:
+    from huggingface_hub import EvalResult, ModelCard, ModelCardData
+
+    model_name = model_repo.split("/")[-1]
+    k = mcfg.num_labels
+    labels = labels_for(k)
+    ms = _metrics_summary(metrics) if metrics else None
+    m3 = _metrics3_summary(metrics3) if metrics3 else None
+    scratch = is_scratch_init(ckpt_cfg)
+    if scratch and training is not None:
+        raise AssertionError("scratch-initialised checkpoint must not carry a pretraining summary")
+    man = manifest or {}
+    totals = (man.get("counts") or {}).get("totals") or {}
+    tr_rows, va_rows, te = totals.get("finetune_train") or {}, totals.get("finetune_val") or {}, totals.get("test_holdout") or {}
+    te3 = totals.get("test_holdout3") or {}
+    pt_man = man.get("pretrain") or {}
+    decon = (((man.get("drops") or {}).get("decontamination_vs_holdout") or {}).get("by_reason") or {}).get("total")
+    holdout_src = man.get("holdout_test_source") or (cfg or {}).get("data", {}).get("holdout_test_source") or "TREC-06"
+    te_rate = f"{100 * te['base_rate']:.1f}% positive" if te.get("base_rate") is not None else "positive rate per metrics.json"
+    eval_results = []
+    if ms and ms.get("split") == "test" and not ms.get("dev_run"):
+        eval_results += [
+            EvalResult(task_type="text-classification", dataset_type=DATASET_IDS[0], dataset_name="TREC-06 holdout (puyang2025), binary roll-up",
+                       metric_type=mt, metric_name=mn, metric_value=round(float(ms[kk]), 4))
+            for kk, mt, mn in (("recall", "recall", "Recall (unwanted)"), ("precision", "precision", "Precision (unwanted)"),
+                               ("f1", "f1", "F1 (unwanted)"), ("accuracy", "accuracy", "Accuracy"), ("roc_auc", "roc_auc", "ROC-AUC"))
+            if ms.get(kk) is not None
+        ]
+    if m3 and m3.get("split") == "test3" and not m3.get("dev_run"):
+        eval_results += [
+            EvalResult(task_type="text-classification", dataset_type=AURA_DATASET_ID,
+                       dataset_name="three-way holdout (Nazario + Nigerian Fraud phishing, SpamAssassin / Enron-Spam spam, CEAS-08 / TREC-07 / SpamAssassin ham)",
+                       metric_type=mt, metric_name=mn, metric_value=round(float(m3[kk]), 4))
+            for kk, mt, mn in (("macro_f1", "f1", "Macro-F1 (legitimate / phishing / spam)"), ("accuracy", "accuracy", "Accuracy (3-way)"))
+            if m3.get(kk) is not None
+        ]
+    datasets = list(DATASET_IDS) + ([AURA_DATASET_ID] if k == 3 else [])
+    data = ModelCardData(language="en", license="mit", library_name="pytorch", pipeline_tag="text-classification",
+                         tags=list(CARD_TAGS) + (["spam-detection"] if k == 3 else []), datasets=datasets, model_name=model_name,
+                         eval_results=eval_results or None)
+    ft = ckpt_cfg.get("finetune") or (cfg or {}).get("finetune") or {}   # the checkpoint's own dials win
+    pt = (cfg or {}).get("pretrain") or {}
+    size_md = f"~{weight_bytes / 1e6:,.0f} MB"
+
+    # Only reachable with --allow-metrics-mismatch: say on the card itself that the numbers were not produced on this checkpoint.
+    def mismatch_md(problems: Sequence[str]) -> str:
+        return (f"> **Warning:** this table was exported with `--allow-metrics-mismatch`; the metrics file does not belong to the "
+                f"exported checkpoint ({'; '.join(problems)}).\n\n") if problems else ""
+
+    if ms:
+        metrics_md = mismatch_md(metrics_mismatch) + _binary_table_md(ms, metrics or {})
+    else:
+        metrics_md = "Not yet evaluated: run `src/evaluate.py run --checkpoint ... --split test` and re-export to fill this table."
+    if k == 3:
+        if m3:
+            metrics3_md = mismatch_md(metrics3_mismatch) + _threeway_table_md(m3)
+        else:
+            metrics3_md = ("Not yet evaluated: run `src/evaluate.py run --checkpoint ... --split test3` (writes `reports/metrics_test3.json`) "
+                           "and re-export to fill this table. " + LABEL_PROVENANCE)
+        metrics3_section = f"\n### Three-way holdout (legitimate / phishing / spam)\n\n{metrics3_md}\n"
+        classes_md = (f"to three classes: `0 = Legitimate`, `1 = Phishing`, `2 = Spam`. The binary verdict shown by the demo is the roll-up "
+                      f"**{UNWANTED_NAME}** = P(phishing) + P(spam) (`config.json[\"binary_rollup\"]`). {LABEL_PROVENANCE}")
+    else:
+        metrics3_section = ""
+        classes_md = (f"to two classes: `0 = Legitimate`, `1 = {UNWANTED_NAME}` (phishing **or** spam; the public corpora do not separate the two).")
+
+    if scratch:
+        pretrain_md = (f"1. **No masked-language-model pretraining.** This checkpoint was fine-tuned from random initialisation "
+                       f"(`finetune.init = scratch`); it is the from-scratch control in the project's experiments comparison, so the "
+                       f"encoder has seen only the labelled fine-tuning data below.")
+    else:
+        if training:
+            tr_md = "\n".join(f"   - {kk}: {v}" for kk, v in training.items())
+        else:
+            tr_md = (f"   - pretraining summary not available at export time: the checkpoint's init descriptor "
+                     f"`{finetune_init(ckpt_cfg) or 'pretrained'}` did not resolve to a `final/config.json` and no `--pretrain-config` "
+                     f"was given; see RUN_LOG.md for the run record.")
+        pretrain_md = "\n".join([
+            f"1. **Masked-language-model pretraining from scratch** on the deduplicated Enron email corpus "
+            f"(`{DATASET_IDS[1]}`: {_n(pt_man.get('rows_in'))} rows -> {_n(pt_man.get('rows_out'))} documents after exact/near-duplicate "
+            f"removal and quote stripping), {int(100 * pt.get('mlm_probability', 0.15))}% dynamic masking "
+            f"({'/'.join(str(int(100 * x)) for x in pt.get('mask_replace_probs', (0.8, 0.1, 0.1)))}), sequence packing to "
+            f"{mcfg.max_seq_len} tokens, AdamW (betas {tuple(pt.get('betas', (0.9, 0.98)))}, weight decay {pt.get('weight_decay', 0.01)}), "
+            f"peak LR {pt.get('lr', '1.5e-4')}, bf16 autocast on one RTX 3090 Ti. Run summary (from the pretraining run's "
+            f"`final/config.json`):",
+            tr_md])
+    ft_md = "\n".join(f"   - {kk}: {v}" for kk, v in ft.items()) if ft else "   - see config.yaml `finetune`"
+    if k == 3:
+        ft_sources = (f"TREC-05, TREC-07 and CEAS-08 (positives trained as *unwanted* only), Enron-Spam and SpamAssassin (spam folders = `Spam`), "
+                      f"the Nazario and Nigerian-fraud corpora of `{AURA_DATASET_ID}` (= `Phishing`) and synthetic BITS-style lures (= `Phishing`)")
+        loss_md = ("Rows with a clean three-way label train with 3-class cross-entropy; TREC / CEAS positives train on the probability mass "
+                   "of {phishing, spam} (partial-label loss: -log(P(phishing) + P(spam))), so they shape the legitimate-vs-unwanted decision "
+                   "without asserting a kind they were never labelled with.")
+        te3_md = (f" A second holdout, **test_holdout3** ({_n(te3.get('rows'))} rows), holds clean-provenance three-way rows never seen by the "
+                  f"tokenizer or the pretraining corpus for the 3x3 confusion matrix.") if te3 else (
+                  " A second holdout, **test_holdout3**, holds clean-provenance three-way rows never seen by the tokenizer or the pretraining "
+                  "corpus for the 3x3 confusion matrix.")
+    else:
+        ft_sources = "TREC-05, TREC-07, CEAS-08, Enron-Spam and SpamAssassin rows"
+        loss_md = ""
+        te3_md = ""
+    lat_ms = (ms or {}).get("latency_ms") or {}
+    lat_dev = " — DEV RUN numbers." if (ms or {}).get("dev_run") else "."
+    lat_head = "Measured single-email latency (batch 1, tokenize + forward + softmax, see the metrics table): "
+    if lat_ms.get("gpu_bf16_avg") is not None:
+        latency_md = (lat_head + f"GPU bf16 {lat_ms['gpu_bf16_avg']} ms avg on "
+                      f"{(metrics or {}).get('latency', {}).get('gpu_bf16', {}).get('device', 'the evaluation GPU')}"
+                      + (f", CPU fp32 ({lat_ms.get('cpu_threads')} threads) {lat_ms['cpu_fp32_avg']} ms avg" if lat_ms.get("cpu_fp32_avg") is not None else "")
+                      + lat_dev)
+    elif lat_ms.get("cpu_fp32_avg") is not None:
+        # CPU-only evaluation (e.g. the GPU was busy): evaluate.py HAS run, so say what was measured
+        latency_md = (lat_head + f"CPU fp32 ({lat_ms.get('cpu_threads')} threads) {lat_ms['cpu_fp32_avg']} ms avg; GPU latency was not "
+                      "measured in this evaluation (CPU-only run)" + lat_dev)
+    else:
+        latency_md = "Single-email latency is reported in the metrics table once `src/evaluate.py` has run."
+    rollup_py = ("p_unwanted = sum(float(probs[i]) for i in cfg[\"binary_rollup\"][\"unwanted\"])   # P(phishing) + P(spam)\n"
+                 "print(\"unwanted\" if p_unwanted >= 0.5 else \"legitimate\", {cfg[\"labels\"][str(i)]: round(float(p), 4) for i, p in enumerate(probs)})")
+    files_cfg = ("model config, num_labels, labels, binary_rollup, label provenance, max_seq_len, export dtype, source checkpoint, fine-tune init, "
+                 "binary metrics (TREC-06), three-way metrics (test_holdout3), pretraining summary (null for scratch init)")
+
+    body = f"""
+# {model_name}: {'legitimate / phishing / spam' if k == 3 else 'phishing'} email classifier trained from scratch
+
+> **No external pretrained weights were used.** The encoder, the tokenizer and every parameter in `model.safetensors`
+> were trained from random initialisation inside this project, in plain PyTorch, without the `transformers`
+> library. The only external inputs are the public email corpora listed below.
+> Initialisation of this checkpoint: `{finetune_init(ckpt_cfg) or 'not recorded'}`.
+
+## What it is
+
+A {params:,}-parameter BERT-style **pre-LayerNorm Transformer encoder** ({mcfg.n_layers} layers, d_model {mcfg.d_model},
+{mcfg.n_heads} heads, FFN {mcfg.d_ff}, byte-level BPE vocabulary of {mcfg.vocab_size:,}, {mcfg.max_seq_len} positions) with a
+CLS-pooled classification head that maps an email (`subject + "\\n\\n" + body`) {classes_md}
+The architecture is defined in `model.py`, which ships with the weights. Weights are stored in **{dtype}** safetensors
+({size_md}); the header carries the architecture config and the label names as JSON.
+
+## How it was trained
+
+{pretrain_md}
+2. **Fine-tuning** on `{DATASET_IDS[0]}` with our own split: {ft_sources} form train ({_n(tr_rows.get('rows'))}) / val ({_n(va_rows.get('rows'))}), grouped by
+   near-duplicate cluster so no template straddles the two; **{holdout_src} ({_n(te.get('rows'))} rows, {te_rate}) is held out
+   entirely** as the unseen binary test set and never touched by the tokenizer, pretraining or fine-tuning ({_n(decon)} pool rows that
+   near-duplicate {holdout_src} templates were removed).{te3_md} Counts come from `data/processed/splits_manifest.json`.
+   {loss_md}
+   Fine-tune dials (from the checkpoint's config.json):
+{ft_md}
+
+## Metrics
+
+### {holdout_src} binary holdout ({UNWANTED_NAME})
+
+{metrics_md}
+{metrics3_section}
+## Intended use
+
+A **defensive triage aid for small-business inboxes**: rank or flag incoming mail for a human to review, surface a
+probability and the attention-highlighted tokens that drove it, and shorten the time between a phishing wave
+landing and someone noticing. It is a classroom/final-project model and is not a substitute for a mail gateway.
+
+## Limitations
+
+- **Era**: the labelled data is 1998-2008 spam-track mail (TREC, CEAS, SpamAssassin, Enron){', plus Nazario phishing (1995-2022) and Nigerian-fraud mail' if k == 3 else ''}.
+  Modern lures (OAuth consent phishing, QR codes, MFA fatigue, SaaS-brand impersonation) are under-represented; expect lower recall on them.
+- **Labels follow corpus provenance**: {LABEL_PROVENANCE if k == 3 else 'the public corpora label "unwanted" mail, not phishing specifically; the credential-lure slice in the metrics is the closest proxy for phishing-proper recall.'}
+  {'A phishing-vs-spam confusion therefore partly measures corpus style, and the credential-lure slice remains the closest proxy for phishing-proper recall.' if k == 3 else ''}
+- **Length confound**: legitimate emails in these corpora average ~2x the tokens of positives. Inputs are truncated to
+  {mcfg.max_seq_len} tokens, and the error analysis checks whether errors follow length, but a short legitimate note can
+  still look suspicious.
+- **English only**; byte-level BPE will tokenize other scripts but the model has not seen them.
+- **Headers are not used** (TREC rewrote them); the model sees subject and body text only. Sender reputation,
+  SPF/DKIM and URL reputation are out of scope and should be combined with this score in any real deployment.
+- Attention highlights are a signal of what the model attended to, not an explanation of why it decided.
+
+## How to load
+
+`model.py` from this repository is the only code dependency (plain PyTorch).
+
+```python
+import json, torch
+from safetensors.torch import load_file
+from tokenizers import Tokenizer
+from model import EncoderConfig, EncoderForClassification   # model.py, shipped with the weights
+
+cfg = json.load(open("config.json"))
+model = EncoderForClassification(EncoderConfig.from_dict(cfg["model"]))   # num_labels = {k}
+model.load_state_dict(load_file("model.safetensors"), strict=True)   # bf16 weights cast into the fp32 module
+model.eval()
+
+tok = Tokenizer.from_file("tokenizer.json")
+tok.enable_truncation(max_length=cfg["max_seq_len"])
+enc = tok.encode("Your account has been suspended\\n\\nClick here to verify your identity within 24 hours.")
+ids = torch.tensor([enc.ids])
+mask = torch.ones_like(ids, dtype=torch.bool)
+with torch.inference_mode():
+    probs = model(ids, mask).float().softmax(-1)[0]
+{rollup_py}
+```
+
+On a GPU wrap the forward in `torch.autocast("cuda", dtype=torch.bfloat16)`. {latency_md}
+
+## Files
+
+- `model.safetensors` — {dtype} weights, metadata `format=pt, dtype={dtype}, architecture=EncoderForClassification, config=<json>, labels=<json>`
+- `config.json` — {files_cfg}
+- `tokenizer.json` — byte-level BPE, NFKC, case preserved, specials `[PAD]=0 [UNK]=1 [CLS]=2 [SEP]=3 [MASK]=4`
+- `README.md` — this card
+
+## Provenance
+
+Built for the Skillspire AI & Machine Learning course final project (BITS, 2026). Data decisions, measured VRAM /
+throughput / latency and every challenge hit along the way are recorded in the project's RUN_LOG.md.
+"""
+    card = ModelCard(f"---\n{data.to_yaml()}\n---\n{body}")
+    return card.content
+
+
+# ------------------------------------------------------------------------------- app
+def copy_to_app(out_dir: Path, app_dir: Path, src_model_py: Path) -> list[Path]:
+    """model.py byte-for-byte (hash asserted) plus the weight files. app.py / requirements.txt
+    belong to the app owner and are never touched."""
+    app_dir.mkdir(parents=True, exist_ok=True)
+    written: list[Path] = []
+    dst = app_dir / "model.py"
+    shutil.copyfile(src_model_py, dst)
+    h_src, h_dst = common.sha1_hex(src_model_py.read_bytes().decode("utf-8")), common.sha1_hex(dst.read_bytes().decode("utf-8"))
+    if h_src != h_dst or src_model_py.read_bytes() != dst.read_bytes():
+        raise RuntimeError(f"{dst} is not byte-identical to {src_model_py}")
+    LOG.info("copied %s -> %s (sha1 %s)", src_model_py, dst, h_src)
+    written.append(dst)
+    for name in WEIGHT_FILES:
+        if name in PROTECTED_APP_FILES:  # defensive: never overwrite another owner's files
+            continue
+        shutil.copyfile(out_dir / name, app_dir / name)
+        written.append(app_dir / name)
+    return written
+
+
+def tree(paths: Sequence[Path], root: Path) -> str:
+    lines = [f"{root}/"]
+    for p in sorted(paths):
+        size = p.stat().st_size
+        unit = f"{size / 1e6:,.1f} MB" if size >= 1e6 else f"{size / 1e3:,.1f} kB"
+        lines.append(f"  {p.name:<20} {unit:>12}")
+    return "\n".join(lines)
+
+
+def check_metrics_match(m: dict | None, path: Path, what: str, ckpt: Path, k: int, expect_split: str, allow: bool) -> list[str]:
+    """Refuse to quote another run's numbers on this checkpoint's card; returns the list of mismatches.
+
+    evaluate.py writes every metrics file under a fixed name in reports/, so evaluating the
+    cross_entropy / weighted / focal runs in sequence leaves the LAST run's numbers there, and an
+    export of the best run with the default paths would put those numbers (and, since the card
+    wording follows num_labels, a contradictory head description) on this model's card with no
+    warning. A metrics file is accepted only when its ``checkpoint`` resolves to the exported one,
+    its ``num_labels`` equals the head size and its ``split`` is the one the table describes
+    (``test`` for the TREC-06 binary table, ``test3`` for the three-way table). A file without a
+    ``checkpoint`` field cannot be attributed and counts as a mismatch. With
+    ``--allow-metrics-mismatch`` the file is still used, the mismatch is logged, and
+    ``metrics_checkpoint_matches: false`` is recorded in config.json / export_manifest.json."""
+    if m is None:
+        return []
+    problems: list[str] = []
+    mc = m.get("checkpoint")
+    if mc is None or Path(str(mc)).resolve() != ckpt:
+        problems.append(f"checkpoint {mc!r} != {ckpt}")
+    mk = m.get("num_labels")
+    if mk is None or int(mk) != k:
+        problems.append(f"num_labels {mk!r} != {k}")
+    if m.get("split") != expect_split:
+        problems.append(f"split {m.get('split')!r} != {expect_split!r}")
+    if problems:
+        msg = f"{what}: {path} does not belong to the exported checkpoint: " + "; ".join(problems)
+        if not allow:
+            raise ValueError(msg + " (re-run src/evaluate.py on this checkpoint, point --metrics-json/--metrics3-json at its "
+                                   "metrics files, or pass --allow-metrics-mismatch to export anyway with the mismatch recorded)")
+        LOG.warning("%s; used anyway (--allow-metrics-mismatch), recorded as metrics_checkpoint_matches=false", msg)
+    return problems
+
+
+def _read_metrics(path: Path, what: str) -> dict | None:
+    if not path.exists():
+        LOG.info("%s: %s absent", what, path)
+        return None
+    try:
+        m = common.read_json(path)
+    except (OSError, json.JSONDecodeError) as e:
+        LOG.warning("%s: could not read %s: %s", what, path, e)
+        return None
+    LOG.info("%s: %s (%s split, %s%s)", what, path, m.get("split"), m.get("timestamp"), ", DEV RUN" if m.get("dev_run") else "")
+    return m
+
+
+# -------------------------------------------------------------------------------- main
+def run(args: argparse.Namespace) -> int:
+    cfg = common.load_config(args.config)
+    global LOG
+    LOG = common.setup_logging("export", cfg["paths"]["logs"])
+    common.set_seed(int(cfg["seed"]))
+    ckpt = Path(args.checkpoint).resolve()
+    run_name = ckpt.parent.name if ckpt.name in ("best", "last") else ckpt.name
+    out_dir = Path(args.out_dir) if args.out_dir else common.project_root() / "model_export" / run_name
+    app_dir = Path(args.app_dir) if args.app_dir else Path(cfg["paths"]["app_dir"])
+    out_dir.mkdir(parents=True, exist_ok=True)
+    LOG.info("checkpoint=%s run_name=%s out_dir=%s app_dir=%s dtype=%s", ckpt, run_name, out_dir, app_dir, args.dtype)
+
+    model, mcfg, ckpt_cfg = load_classifier(ckpt)
+    k = mcfg.num_labels
+    LOG.info("labels: %s; binary roll-up: %s", labels_for(k), binary_rollup_for(k))
+    weights = export_weights(model, mcfg, args.dtype, out_dir / "model.safetensors")
+
+    tok_src = Path(cfg["paths"]["tokenizer"])
+    shutil.copyfile(tok_src, out_dir / "tokenizer.json")
+    LOG.info("copied %s (%.1f kB)", tok_src, tok_src.stat().st_size / 1e3)
+
+    reports = Path(cfg["paths"]["reports"])
+    metrics_path = Path(args.metrics_json) if args.metrics_json else reports / "metrics.json"
+    metrics = _read_metrics(metrics_path, "binary metrics (TREC-06)")
+    mismatch = check_metrics_match(metrics, metrics_path, "binary metrics (TREC-06)", ckpt, k, "test", args.allow_metrics_mismatch)
+    if args.metrics3_json:
+        metrics3_path = Path(args.metrics3_json)
+    else:
+        m3cfg = (cfg.get("eval") or {}).get("model_card_metrics3")
+        metrics3_path = (common.project_root() / m3cfg) if m3cfg else reports / "metrics_test3.json"
+    metrics3 = _read_metrics(metrics3_path, "three-way metrics (test_holdout3)") if k == 3 else None
+    mismatch3 = check_metrics_match(metrics3, metrics3_path, "three-way metrics (test_holdout3)", ckpt, k, "test3", args.allow_metrics_mismatch)
+    if k == 2:
+        LOG.info("2-class checkpoint: three-way metrics table skipped")
+    elif metrics3 and not (metrics3.get("threeway") or {}).get("evaluated"):
+        LOG.warning("%s has no evaluated three-way block (%s); card table left as 'not yet evaluated'", metrics3_path,
+                    (metrics3.get("threeway") or {}).get("reason"))
+    training, training_path = find_pretrain_summary(ckpt_cfg, args.pretrain_config)
+    LOG.info("fine-tune init: %s; pretraining summary: %s", finetune_init(ckpt_cfg), training_path or ("none (scratch init)" if is_scratch_init(ckpt_cfg) else "absent"))
+    manifest = read_splits_manifest(Path(cfg["paths"]["data_processed"]))
+    LOG.info("splits manifest: %s", "loaded" if manifest else "absent (card quotes n/a for dataset counts)")
+
+    export_cfg = build_config(mcfg, ckpt, ckpt_cfg, args.dtype, metrics, str(metrics_path) if metrics else None,
+                              metrics3, str(metrics3_path) if metrics3 else None, training, training_path,
+                              metrics_mismatch=mismatch, metrics3_mismatch=mismatch3)
+    common.write_json(export_cfg, out_dir / "config.json")
+    card = model_card(mcfg, cfg, ckpt_cfg, metrics, metrics3, training, args.dtype, weights["params"], weights["bytes"], cfg["hf"]["model_repo"], manifest,
+                      metrics_mismatch=mismatch, metrics3_mismatch=mismatch3)
+    (out_dir / "README.md").write_text(card, encoding="utf-8")
+    LOG.info("wrote %s (%d chars)", out_dir / "README.md", len(card))
+
+    app_written = copy_to_app(out_dir, app_dir, Path(__file__).resolve().parent / "model.py")
+
+    summary = {
+        "timestamp": datetime.now().isoformat(timespec="seconds"), "run_name": run_name, "checkpoint": str(ckpt),
+        "out_dir": str(out_dir), "app_dir": str(app_dir), "dtype": args.dtype, "weights": weights,
+        "num_labels": k, "labels": labels_for(k), "binary_rollup": binary_rollup_for(k),
+        "files": {p.name: p.stat().st_size for p in out_dir.iterdir()},
+        "app_files": {p.name: p.stat().st_size for p in app_written},
+        "model_py_sha1": common.sha1_hex((Path(__file__).resolve().parent / "model.py").read_text(encoding="utf-8")),
+        "metrics_included": metrics is not None, "metrics_threeway_included": bool(metrics3 and export_cfg["metrics_threeway"]),
+        "metrics_checkpoint_matches": export_cfg["metrics_checkpoint_matches"], "metrics_mismatch": export_cfg["metrics_mismatch"],
+        "metrics_threeway_checkpoint_matches": export_cfg["metrics_threeway_checkpoint_matches"], "metrics_threeway_mismatch": export_cfg["metrics_threeway_mismatch"],
+        "training_included": training is not None,
+        "finetune_init": finetune_init(ckpt_cfg), "manifest_included": manifest is not None,
+    }
+    common.write_json(summary, out_dir / "export_manifest.json")
+    print("\n" + tree(list(out_dir.iterdir()), out_dir))
+    print(tree(app_written, app_dir))
+    print(f"\nmodel.py sha1 {summary['model_py_sha1']} (src == app, asserted); weights {weights['bytes'] / 1e6:,.1f} MB {args.dtype}, "
+          f"{weights['params']:,} params; num_labels {k} {labels_for(k)}; init {finetune_init(ckpt_cfg)}; "
+          f"binary metrics {'included' if metrics else 'absent'}; three-way metrics "
+          f"{'included' if summary['metrics_threeway_included'] else ('n/a (2-class)' if k == 2 else 'absent')}; "
+          f"pretraining summary {'included' if training else ('n/a (scratch)' if is_scratch_init(ckpt_cfg) else 'absent')}")
+    return 0
+
+
+def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--config", default="config.yaml")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    r = sub.add_parser("run", help="export a fine-tune best/ dir to model_export/<run>/ and app/")
+    r.add_argument("--checkpoint", required=True, help="fine-tune best/ dir with model.safetensors + config.json")
+    r.add_argument("--out-dir", default=None, help="default model_export/<run_name>")
+    r.add_argument("--app-dir", default=None, help="default config.paths.app_dir")
+    r.add_argument("--dtype", choices=sorted(DTYPES), default="bf16")
+    r.add_argument("--metrics-json", default=None, help="binary TREC-06 metrics; default <reports>/metrics.json")
+    r.add_argument("--metrics3-json", default=None, help="three-way holdout metrics; default config eval.model_card_metrics3 or <reports>/metrics_test3.json")
+    r.add_argument("--pretrain-config", default=None,
+                   help="pretrain final/config.json (default: the checkpoint's pretrained:<path> descriptor; none for init=scratch)")
+    r.add_argument("--allow-metrics-mismatch", action="store_true",
+                   help="export even if a metrics file names another checkpoint / num_labels / split (logged; recorded as "
+                        "metrics_checkpoint_matches=false in config.json and export_manifest.json). Default: refuse.")
+    return ap.parse_args(argv)
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    return run(parse_args(argv))
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
diff --git a/presentation/src/finetune.py b/presentation/src/finetune.py
new file mode 100644
index 0000000000000000000000000000000000000000..790423a87cefcae0f3fbffa790a442820eab2708
--- /dev/null
+++ b/presentation/src/finetune.py
@@ -0,0 +1,1650 @@
+"""Phase 2: fine-tune ``EncoderForClassification`` -> legitimate / phishing / spam.
+
+The instructor's "experimentation dials" (number of layers, epochs, loss function, plus
+learning rate, batch size, frozen layers, synthetic data) are first-class CLI flags, and
+every run leaves ONE JSON line in reports/experiments.jsonl so the experimentation table
+(``finetune.py list``) is generated from evidence rather than typed up afterwards.
+
+Subcommands::
+
+    train [--run-name R] [--init pretrained:<dir>|scratch] [--n-layers N] [--freeze-layers K]
+          [--loss cross_entropy|weighted_cross_entropy|focal] [--focal-gamma G] [--lr LR]
+          [--batch-size B] [--micro-batch M] [--epochs E] [--max-steps S] [--eval-every S]
+          [--log-every S] [--synthetic on|off] [--limit-rows N] [--num-labels 2|3] [--cpu]
+          [--resume] [--overwrite]
+    plot  --run-name R [--out PNG]       reports/finetune_curves.png
+    list                                  experiments.jsonl as a markdown table
+
+ML decisions, in brief:
+
+* **Three classes, partial labels (RUN_LOG entry 6).** ``finetune.num_labels`` (3) sizes the
+  classification head; config ``model:`` is never touched because the live pretraining run
+  refuses to resume if its model snapshot changes. Every row carries ``label3`` in
+  {0 legitimate, 1 phishing, 2 spam, -1 unwanted-but-unsplit} and ``label_set`` in
+  {legit, phish, spam, unwanted}; the binary ``label`` column (1 = phishing OR spam) stays.
+  The TREC-05/07 and CEAS-08 positives never separated spam from phishing, so they train on
+  the probability MASS of {phishing, spam}: per row an allowed-class mask ``[B, 3]`` built
+  from label_set, and the loss is the partial-label negative log-likelihood
+  ``logsumexp(all logits) - logsumexp(logits[allowed])`` = -log sum_{c in allowed} p_c
+  (Jin & Ghahramani 2002; Cour et al. 2011). For a single allowed class this IS
+  ``F.cross_entropy``; for an "unwanted" row it is ``-log(p_phish + p_spam)``, which still
+  shapes the legitimate-vs-unwanted decision without asserting which kind it was. With
+  ``num_labels: 2`` the old binary behaviour is reproduced exactly (label3 ignored; the mask
+  is the one-hot label) so the binary baseline runs remain reproducible.
+* **Two metric sets.** (a) BINARY roll-up on all val rows: P(unwanted) = p1 + p2 vs the
+  binary label at threshold 0.5 -> accuracy/precision/recall/F1/ROC-AUC/PR-AUC with the same
+  definitions as the binary runs (so results stay comparable). (b) THREE-WAY on rows with
+  label3 in {0, 1, 2}: argmax over 3 classes -> 3x3 confusion, per-class P/R/F1, macro-F1
+  over all three classes, plus a phishing-vs-spam confusion restricted to true unwanted rows
+  (argmax over {phishing, spam} only: "given it is unwanted, which kind?"). Display names
+  are {0: Legitimate, 1: Phishing, 2: Spam}; the roll-up is "Unwanted (phishing or spam)".
+  Selection / early stopping follow ``finetune.early_stopping.metric``: ``val_f1_unwanted``
+  (default; = today's binary F1) or ``val_macro_f1_3way``.
+
+* **Data path.** Train = finetune_train.parquet (+ optional synthetic rows), selection =
+  finetune_val.parquet. test_holdout.parquet (TREC-06) is refused by file name AND by the
+  ``source`` column of anything loaded, so a misconfigured path cannot leak the holdout.
+  Texts are encoded once with ``tokenizer.encode_batch`` (truncate to model.max_seq_len,
+  pad to a multiple of finetune.dynamic_padding_multiple), stored trimmed, and re-padded per
+  batch with the identical rule; the first batch is checked tensor-for-tensor against a
+  direct ``encode_batch`` call so the app / evaluate.py path (which calls ``encode_batch``
+  per request) sees exactly the inputs the classifier was trained on. No sequence packing:
+  the bool key-padding mask takes SDPA's memory-efficient kernel (research_model-math.md sec. 5).
+* **DataLoader.** Every batch goes through ``torch.utils.data.DataLoader`` (``make_loader``):
+  a row-index ``Dataset``, the seeded length-bucketed batch order as ``batch_sampler`` and a
+  collate function that pads per micro-batch. Shuffling, batching and preprocessing therefore
+  live in the standard PyTorch data pipeline the course reference uses.
+* **Length bucketing.** Each epoch: seeded permutation -> mega-batches of 50 x batch_size ->
+  sort by token length inside each -> cut into batches -> shuffle the batch order. Padding
+  waste drops from ~50% (random batches over a 20..512 length spread) to a few percent while
+  the batch composition stays random at the mega-batch level. The order is a pure function
+  of (seed, epoch) so ``--resume`` can skip exactly the batches already consumed.
+* **Init.** ``pretrained:<dir>`` loads ``encoder.*`` from a Phase 1 checkpoint with strict
+  key matching (``EncoderForClassification.from_pretrained_encoder``); ``scratch`` is the
+  ablation that shows what MLM pretraining buys. ``--n-layers N`` keeps the FIRST N encoder
+  layers (``encoder.layers.0..N-1``) of the pretrained stack and drops the rest; the final
+  LayerNorm is kept although it was trained on layer-24 output (fine-tuning re-fits it).
+  ``--freeze-layers K`` freezes embeddings + layers 0..K-1: cheaper steps (autograd stops at
+  layer K) and less catastrophic forgetting on a small train set.
+* **Loss dial.** All three variants are defined over the partial-label NLL above (``nll``):
+  cross_entropy = mean(nll); weighted_cross_entropy = sum(w_r nll_r) / sum(w_r) with
+  inverse-frequency class weights w_c = N / (C * n_c) from the fully-labelled rows of the
+  train split actually used and w_r = mean of w over the row's allowed classes (so an
+  "unwanted" row gets mean(w_phish, w_spam)) -- the sum-of-weights normalisation is
+  ``F.cross_entropy(weight=)``'s, so the two agree on fully-labelled batches; focal loss
+  (Lin et al. 2017) ``(1 - exp(-nll))^gamma * nll`` = ``-(1 - p_t)^gamma log p_t`` without
+  alpha, where p_t is the allowed-mass. Loss is computed in fp32 on bf16-autocast logits.
+* **Optimizer / schedule.** The same ``build_optimizer`` (AdamW, no weight decay on biases
+  and LayerNorm, fused) and the same fwd/bwd -> clip 1.0 -> step -> zero_grad(set_to_none)
+  pattern the smoke test measured. Linear warmup over finetune.warmup_fraction of the total
+  optimizer steps, then linear decay to 0 (BERT fine-tuning schedule). Gradient
+  accumulation: a logical batch of ``--batch-size`` rows is split into ``--micro-batch``
+  chunks (default 16, the smoke test's 13.65 GiB point for 16 x 512 with the heavier MLM
+  head); each chunk's loss is scaled by its row share so the step gradient is the mean over
+  the logical batch even when the last batch of an epoch is short.
+* **Selection / early stopping.** Full validation every ``--eval-every`` steps and at every
+  epoch end; best/ is rewritten whenever the selection metric (val F1 of the unwanted
+  roll-up at threshold 0.5 by default, or 3-way macro-F1) improves;
+  finetune.early_stopping.patience evaluations without improvement stop the run. The
+  F1-optimal threshold on val is recorded next to the 0.5 decision threshold so evaluate.py
+  / the app can show both; selection itself uses 0.5 to stay comparable across runs.
+  best/config.json carries num_labels, the label names, the selection metric, both metric
+  sets and the thresholds.
+* **--cpu (dev only).** Runs the identical code path on the CPU with whatever config it is
+  given (tiny encoder + fixture parquet files) so the whole script can be exercised while
+  the GPU is busy; autocast and the VRAM guard are disabled and torch.cuda is never initialised:
+  device info, allocator stats and the CUDA RNG snapshot are gated on the chosen device, and
+  CUDA_VISIBLE_DEVICES is cleared for the process before the runtime is first queried (torch
+  2.14's ``AdamW.step`` health check would otherwise create a context on any visible GPU). If a
+  device is still visible after that, the run refuses to start. The default path still requires CUDA.
+* **Resume.** last/ holds model + trainer_state.pt (optimizer, scheduler, step, epoch,
+  batch-in-epoch, Python/NumPy/torch CPU/torch CUDA RNG states, best score, patience counter,
+  cumulative seconds, the resolved dials). ``--resume`` restores all of it and skips the
+  already-consumed batches of the current epoch. Dials that shape the optimizer state or the
+  batch order (init, n_layers, freeze_layers, batch_size, seed) are refused on resume; ``--lr``
+  is re-applied to the restored optimizer/scheduler so the record never claims an lr that was
+  not used; metrics.csv / epochs.csv rows logged after the checkpoint (a crash between
+  checkpoints) are dropped before appending; a run already at its --max-steps / --epochs is
+  reported as "nothing to resume" without training an extra step. SIGINT/SIGTERM save last/
+  and exit 0.
+* **VRAM guard.** ``PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`` before CUDA starts;
+  after every optimizer step the reserved peak is compared with config.vram.hard_cap_gb
+  (GiB, the smoke test's convention) and the run aborts (after saving last/) on a breach.
+"""
+
+from __future__ import annotations
+
+import os
+
+os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
+
+import argparse  # noqa: E402
+import contextlib  # noqa: E402
+import csv  # noqa: E402
+import json  # noqa: E402
+import logging  # noqa: E402
+import math  # noqa: E402
+import random  # noqa: E402
+import signal  # noqa: E402
+import sys  # noqa: E402
+import time  # noqa: E402
+import zlib  # noqa: E402
+from dataclasses import asdict, dataclass, field, fields  # noqa: E402
+from pathlib import Path  # noqa: E402
+from typing import Any  # noqa: E402
+
+sys.path.insert(0, str(Path(__file__).parent))
+
+import numpy as np  # noqa: E402
+import pyarrow.parquet as pq  # noqa: E402
+import torch  # noqa: E402
+import torch.nn.functional as F  # noqa: E402
+from safetensors import safe_open  # noqa: E402
+from safetensors.torch import load_file, save_file  # noqa: E402
+from torch import Tensor  # noqa: E402
+from torch.utils.data import DataLoader, Dataset  # noqa: E402
+
+import common  # noqa: E402
+from model import EncoderConfig, EncoderForClassification, count_parameters  # noqa: E402
+from smoke_test import build_optimizer  # noqa: E402  (the measured optimizer code path)
+from tokenizer import encode_batch, load_tokenizer  # noqa: E402
+
+LOG = logging.getLogger("finetune")
+GIB = float(2**30)
+
+TRAIN_FILE = "finetune_train.parquet"
+VAL_FILE = "finetune_val.parquet"
+SYNTHETIC_FILE = "synthetic.parquet"
+HOLDOUT_FILE = "test_holdout.parquet"
+LOSSES = ("cross_entropy", "weighted_cross_entropy", "focal")
+MEGA_BATCH_FACTOR = 50
+ENCODE_CHUNK = 2048
+DECISION_THRESHOLD = 0.5
+METRIC_KEYS = ("accuracy", "precision", "recall", "f1", "roc_auc", "pr_auc")
+RUN_LOG_MIN_STEPS = 100
+# Dials whose change on --resume would leave the restored optimizer state / batch order / head shape inconsistent.
+RESUME_FIXED_DIALS = ("init", "n_layers", "freeze_layers", "batch_size", "seed", "num_labels")
+# The only implemented values of the config-only dials; anything else in config.yaml is an error, not a no-op.
+IMPLEMENTED = {"schedule": "linear_decay", "pooling": "cls"}
+CONFIG_PATH = {"schedule": "finetune.schedule", "pooling": "finetune.pooling",
+               "selection_metric": "finetune.early_stopping.metric", "num_labels": "finetune.num_labels"}
+
+# ---- three-way label contract (RUN_LOG entry 6) -----------------------------------------------
+LABEL_NAMES = {0: "Legitimate", 1: "Phishing", 2: "Spam"}
+UNWANTED_NAME = "Unwanted (phishing or spam)"
+LABEL_SETS: dict[str, tuple[int, ...]] = {"legit": (0,), "phish": (1,), "spam": (2,), "unwanted": (1, 2)}
+LABEL3_OF_SET = {"legit": 0, "phish": 1, "spam": 2, "unwanted": -1}
+NUM_LABELS_ALLOWED = (2, 3)
+# Selection metrics. val_f1_unwanted is today's binary F1 (p_unwanted = p1 + p2 at 0.5) so results stay
+# comparable with the binary runs; val_f1_phishing is the pre-entry-6 name of the same number and is
+# accepted so the recorded binary baselines resolve. val_macro_f1_3way needs num_labels == 3.
+SELECTION_METRICS = ("val_f1_unwanted", "val_macro_f1_3way", "val_f1_phishing")
+SELECTION_KEY = {"val_f1_unwanted": "f1", "val_f1_phishing": "f1", "val_macro_f1_3way": "macro_f1_3way"}
+CLASS_SLUGS = ("legitimate", "phishing", "spam")
+CM3_KEYS = tuple(f"cm3_{a[0]}{b[0]}" for a in CLASS_SLUGS for b in CLASS_SLUGS)   # cm3_ll ... cm3_ss (true, pred)
+PVS_KEYS = ("pvs_phish_as_phish", "pvs_phish_as_spam", "pvs_spam_as_phish", "pvs_spam_as_spam")
+THREE_WAY_KEYS = ("n_3way", "accuracy_3way", "macro_f1_3way",
+                  *(f"{m}_{s}" for s in CLASS_SLUGS for m in ("precision", "recall", "f1")),
+                  *CM3_KEYS, "n_unwanted_3way", *PVS_KEYS)
+# validated categorical palette (dataviz skill): blue, orange, aqua, yellow
+PALETTE = ("#2a78d6", "#eb6834", "#1baf7a", "#eda100")
+
+
+# ---------------------------------------------------------------------------------- dials
+@dataclass
+class TrainArgs:
+    """Every dial of a run, resolved from CLI > saved state (on --resume) > config.yaml.
+
+    Stored verbatim in trainer_state.pt and experiments.jsonl so a run is reproducible from
+    its record alone.
+    """
+
+    run_name: str = "finetune"
+    init: str = "pretrained"          # "pretrained" | "pretrained:<path>" | "scratch"
+    n_layers: int = 24
+    freeze_layers: int = 0
+    loss: str = "cross_entropy"
+    focal_gamma: float = 2.0
+    lr: float = 3e-5
+    batch_size: int = 32
+    micro_batch: int = 16
+    epochs: int = 3
+    max_steps: int | None = None
+    eval_every: int = 500
+    log_every: int = 20
+    eval_batch_size: int = 64
+    warmup_fraction: float = 0.06
+    weight_decay: float = 0.01
+    betas: tuple[float, float] = (0.9, 0.98)
+    eps: float = 1e-6
+    grad_clip: float = 1.0
+    patience: int = 2
+    schedule: str = "linear_decay"            # config-only: finetune.schedule
+    selection_metric: str = "val_f1_unwanted" # config: finetune.early_stopping.metric (val_f1_unwanted | val_macro_f1_3way)
+    pooling: str = "cls"                      # config-only: finetune.pooling (EncoderForClassification reads token 0)
+    num_labels: int = 3                       # finetune.num_labels (NOT model.num_labels): 3 = legit/phish/spam, 2 = binary baseline
+    synthetic: str = "off"
+    limit_rows: int | None = None
+    seed: int = 42
+    cpu: bool = False                         # dev only: run the whole script on the CPU
+
+    @classmethod
+    def resolve(cls, ns: argparse.Namespace, cfg: dict, saved: dict | None) -> "TrainArgs":
+        fcfg, mcfg = cfg["finetune"], cfg["model"]
+        pcfg = cfg.get("pretrain", {})
+        base = cls(
+            n_layers=int(mcfg["n_layers"]),
+            # Absent key -> 2 so a config.yaml predating entry 6 still reproduces the binary baseline.
+            num_labels=int(fcfg.get("num_labels", 2)),
+            freeze_layers=int(fcfg.get("freeze_encoder_layers", 0)),
+            loss=str(fcfg.get("loss", "cross_entropy")),
+            focal_gamma=float(fcfg.get("focal_gamma", 2.0)),
+            lr=float(fcfg["lr"]),
+            batch_size=int(fcfg["batch_size"]),
+            micro_batch=int(fcfg.get("micro_batch", 16)),
+            epochs=int(fcfg["epochs"]),
+            eval_every=int(fcfg.get("eval_every_steps", fcfg.get("checkpoint_every_steps", 500))),
+            log_every=int(fcfg.get("log_every_steps", pcfg.get("log_every_steps", 20))),
+            eval_batch_size=int(fcfg.get("eval_batch_size", 64)),
+            warmup_fraction=float(fcfg["warmup_fraction"]),
+            weight_decay=float(fcfg.get("weight_decay", pcfg.get("weight_decay", 0.01))),
+            betas=tuple(fcfg.get("betas", pcfg.get("betas", (0.9, 0.98)))),
+            eps=float(fcfg.get("eps", pcfg.get("eps", 1e-6))),
+            grad_clip=float(fcfg.get("grad_clip", pcfg.get("grad_clip", 1.0))),
+            patience=int(fcfg["early_stopping"]["patience"]),
+            schedule=str(fcfg.get("schedule", IMPLEMENTED["schedule"])),
+            selection_metric=str(fcfg["early_stopping"].get("metric", "val_f1_unwanted")),
+            pooling=str(fcfg.get("pooling", IMPLEMENTED["pooling"])),
+            synthetic="on" if cfg["data"].get("synthetic", {}).get("enabled") else "off",
+            seed=int(cfg["seed"]),
+        )
+        if saved:
+            for k, v in saved.items():
+                if k in {f.name for f in fields(cls)}:
+                    setattr(base, k, tuple(v) if k == "betas" else v)
+        for k, v in vars(ns).items():
+            if k in {f.name for f in fields(cls)} and v is not None and not (k == "cpu" and v is False):
+                setattr(base, k, v)
+        if getattr(ns, "micro_batch", None) is None:
+            # A smaller --batch-size without an explicit --micro-batch means "one micro-batch per step".
+            base.micro_batch = min(base.micro_batch, base.batch_size)
+        for key, allowed in IMPLEMENTED.items():
+            if getattr(base, key) != allowed:
+                raise ValueError(f"config {CONFIG_PATH[key]}={getattr(base, key)!r} is not implemented; only {allowed!r} is "
+                                 f"(editing it would otherwise silently change nothing)")
+        if base.num_labels not in NUM_LABELS_ALLOWED:
+            raise ValueError(f"{CONFIG_PATH['num_labels']} must be one of {NUM_LABELS_ALLOWED}, got {base.num_labels!r}")
+        if base.selection_metric not in SELECTION_METRICS:
+            raise ValueError(f"{CONFIG_PATH['selection_metric']}={base.selection_metric!r} is not implemented; "
+                             f"one of {SELECTION_METRICS[:2]} (val_f1_phishing is the legacy alias of val_f1_unwanted)")
+        if base.selection_metric == "val_macro_f1_3way" and base.num_labels != 3:
+            raise ValueError("selection metric val_macro_f1_3way needs finetune.num_labels: 3")
+        if base.loss not in LOSSES:
+            raise ValueError(f"--loss must be one of {LOSSES}, got {base.loss!r}")
+        if base.batch_size % base.micro_batch:
+            raise ValueError(f"--batch-size {base.batch_size} must be a multiple of --micro-batch {base.micro_batch}")
+        if not 0 <= base.freeze_layers <= base.n_layers:
+            raise ValueError(f"--freeze-layers {base.freeze_layers} outside [0, {base.n_layers}]")
+        return base
+
+    def to_record(self) -> dict[str, Any]:
+        d = asdict(self)
+        d["betas"] = list(self.betas)
+        return d
+
+
+# ----------------------------------------------------------------------------------- data
+@dataclass
+class Labelled:
+    """The label columns of one parquet split after validation against the three-way contract."""
+
+    texts: list[str]
+    labels: np.ndarray          # int64 [n] binary: 1 = phishing OR spam (unchanged from the binary build)
+    label3: np.ndarray          # int64 [n] in {0, 1, 2, -1}; with num_labels == 2 this is a copy of ``labels``
+    allowed: np.ndarray         # bool [n, C] allowed-class mask built from label_set (one-hot for full labels)
+    sources: list[str]
+
+    def __len__(self) -> int:
+        return len(self.texts)
+
+
+@dataclass
+class EncodedSplit:
+    """Token ids for one split, truncated to max_len and stored trimmed (flat int32 + offsets)."""
+
+    name: str
+    flat: np.ndarray            # int32 concatenation of every row's ids
+    offsets: np.ndarray         # int64 [n + 1]
+    labels: np.ndarray          # int64 [n] binary roll-up target (metrics set a)
+    label3: np.ndarray          # int64 [n] three-way target, -1 = unwanted-but-unsplit (metrics set b uses >= 0)
+    allowed: np.ndarray         # bool [n, C] partial-label mask (the loss target)
+    sources: list[str]
+    texts: list[str]            # kept for the encode_batch consistency check only
+
+    def __len__(self) -> int:
+        return int(self.labels.shape[0])
+
+    @property
+    def lengths(self) -> np.ndarray:
+        return np.diff(self.offsets)
+
+    @property
+    def num_labels(self) -> int:
+        return int(self.allowed.shape[1])
+
+    def row(self, i: int) -> np.ndarray:
+        return self.flat[self.offsets[i]:self.offsets[i + 1]]
+
+    def collate(self, idx: np.ndarray, multiple: int, device: torch.device) -> tuple[Tensor, Tensor, Tensor, Tensor]:
+        """Pad to the longest row in the batch rounded up to ``multiple`` (encode_batch's rule).
+
+        Returns (ids int64 [B, L], key-padding mask bool [B, L], binary labels int64 [B],
+        allowed-class mask bool [B, C]).
+        """
+        lens = self.lengths[idx]
+        L = int(lens.max())
+        if multiple:
+            L = int(math.ceil(L / multiple) * multiple)
+        ids = np.zeros((len(idx), L), dtype=np.int64)
+        mask = np.zeros((len(idx), L), dtype=np.bool_)
+        for r, i in enumerate(idx):
+            n = int(lens[r])
+            ids[r, :n] = self.row(int(i))
+            mask[r, :n] = True
+        return (torch.from_numpy(ids).to(device, non_blocking=True),
+                torch.from_numpy(mask).to(device, non_blocking=True),
+                torch.from_numpy(self.labels[idx]).to(device, non_blocking=True),
+                torch.from_numpy(self.allowed[idx]).to(device, non_blocking=True))
+
+
+def allowed_mask(label3: np.ndarray, label_set: list[str], num_labels: int) -> np.ndarray:
+    """Per-row allowed-class mask ``[n, C]`` from label_set (3 classes) or the binary label (2 classes)."""
+    out = np.zeros((len(label3), num_labels), dtype=np.bool_)
+    if num_labels == 2:
+        out[np.arange(len(label3)), label3] = True
+        return out
+    for r, s in enumerate(label_set):
+        out[r, list(LABEL_SETS[s])] = True
+    return out
+
+
+def _assert_not_holdout(path: Path, holdout_source: str) -> None:
+    """Hard rule: TREC-06 is touched only by evaluate.py. Refuse by file name and by source value."""
+    if path.name == HOLDOUT_FILE:
+        raise RuntimeError(f"refusing to load {path.name}: the test holdout is only read by evaluate.py")
+    names = set(pq.read_schema(path).names)
+    if "source" in names:
+        values = {str(v) for v in pq.read_table(path, columns=["source"]).column("source").unique().to_pylist()}
+        norm = holdout_source.lower().replace("_", "-")
+        hits = {v for v in values if v.strip().lower().replace("_", "-") == norm}
+        if hits:
+            raise RuntimeError(f"{path.name} contains holdout source {hits}: test-set leakage")
+
+
+def read_split(path: Path, holdout_source: str, limit: int | None, seed: int, num_labels: int,
+               force_source: str | None = None) -> Labelled:
+    """Read text + labels of one split and validate the three-way contract.
+
+    With ``num_labels == 3`` the ``label3`` / ``label_set`` columns are REQUIRED (a split built
+    before RUN_LOG entry 6 is an error, not a silent all-"unwanted" run) and checked for
+    internal consistency (label_set <-> label3 <-> binary label). With ``num_labels == 2``
+    they are ignored and the binary label is the target, reproducing the binary baseline.
+    """
+    _assert_not_holdout(path, holdout_source)
+    names = set(pq.read_schema(path).names)
+    three = num_labels == 3
+    if three and not {"label3", "label_set"} <= names:
+        raise ValueError(f"{path.name}: finetune.num_labels is 3 but the columns label3/label_set are missing "
+                         f"(have {sorted(names)}); rebuild the split with the three-way data_prep")
+    cols = ["text", "label"] + (["source"] if "source" in names else []) + (["label3", "label_set"] if three else [])
+    table = pq.read_table(path, columns=cols)
+    if limit is not None and table.num_rows > limit:
+        # Seeded random subsample (not a head slice) so a dev run still sees every source.
+        idx = np.sort(np.random.default_rng([seed, zlib.crc32(path.name.encode())]).choice(table.num_rows, size=limit, replace=False))
+        table = table.take(idx)
+    texts = [t or "" for t in table.column("text").to_pylist()]
+    labels = np.asarray(table.column("label").to_pylist(), dtype=np.int64)
+    if not set(np.unique(labels).tolist()) <= {0, 1}:
+        raise ValueError(f"{path.name}: labels must be 0/1, got {np.unique(labels).tolist()}")
+    if force_source is not None:
+        sources = [force_source] * len(texts)
+    else:
+        sources = [str(s) for s in table.column("source").to_pylist()] if "source" in names else ["?"] * len(texts)
+    if not three:
+        return Labelled(texts, labels, labels.copy(), allowed_mask(labels, [], 2), sources)
+
+    label3 = np.asarray(table.column("label3").to_pylist(), dtype=np.int64)
+    label_set = [str(s) for s in table.column("label_set").to_pylist()]
+    bad_sets = sorted(set(label_set) - set(LABEL_SETS))
+    if bad_sets:
+        raise ValueError(f"{path.name}: label_set values outside {sorted(LABEL_SETS)}: {bad_sets}")
+    expected3 = np.fromiter((LABEL3_OF_SET[s] for s in label_set), dtype=np.int64, count=len(label_set))
+    if not np.array_equal(expected3, label3):
+        n_bad = int((expected3 != label3).sum())
+        raise ValueError(f"{path.name}: {n_bad} row(s) where label3 disagrees with label_set "
+                         f"(contract: legit->0, phish->1, spam->2, unwanted->-1)")
+    expected_bin = (label3 != 0).astype(np.int64)
+    if not np.array_equal(expected_bin, labels):
+        raise ValueError(f"{path.name}: {int((expected_bin != labels).sum())} row(s) where the binary label disagrees "
+                         f"with label3 (label must be 1 for phish/spam/unwanted, 0 for legit)")
+    return Labelled(texts, labels, label3, allowed_mask(label3, label_set, 3), sources)
+
+
+def concat_labelled(a: Labelled, b: Labelled) -> Labelled:
+    return Labelled(a.texts + b.texts, np.concatenate([a.labels, b.labels]), np.concatenate([a.label3, b.label3]),
+                    np.concatenate([a.allowed, b.allowed]), a.sources + b.sources)
+
+
+def label_counts(label3: np.ndarray, num_labels: int) -> dict[str, int]:
+    """Human-readable class counts for logs and records (keys: display names + 'unwanted')."""
+    if num_labels == 2:
+        return {LABEL_NAMES[0]: int((label3 == 0).sum()), UNWANTED_NAME: int((label3 == 1).sum())}
+    out = {LABEL_NAMES[c]: int((label3 == c).sum()) for c in range(3)}
+    out["unwanted (unsplit)"] = int((label3 == -1).sum())
+    return out
+
+
+def encode_split(name: str, tok, data: Labelled, max_len: int, multiple: int) -> EncodedSplit:
+    """Encode once with the shared ``encode_batch`` contract, then store rows trimmed of padding."""
+    t0 = common.timer()
+    texts, labels = data.texts, data.labels
+    rows: list[np.ndarray] = []
+    for i in range(0, len(texts), ENCODE_CHUNK):
+        ids, mask = encode_batch(tok, texts[i:i + ENCODE_CHUNK], max_len, pad_to_multiple_of=multiple)
+        n = mask.sum(dim=1).numpy()
+        ids_np = ids.numpy()
+        rows.extend(ids_np[r, :n[r]].astype(np.int32) for r in range(ids_np.shape[0]))
+    lengths = np.fromiter((len(r) for r in rows), dtype=np.int64, count=len(rows))
+    offsets = np.zeros(len(rows) + 1, dtype=np.int64)
+    np.cumsum(lengths, out=offsets[1:])
+    flat = np.concatenate(rows) if rows else np.zeros(0, dtype=np.int32)
+    split = EncodedSplit(name, flat, offsets, labels, data.label3, data.allowed, data.sources, texts)
+    LOG.info("%s: %d rows encoded in %.1fs; tokens mean %.0f / median %.0f / max %d; %.1f%% truncated at %d; "
+             "binary positives %d (%.1f%%); classes %s", name, len(rows), common.timer() - t0, lengths.mean(),
+             np.median(lengths), lengths.max(), 100.0 * float((lengths >= max_len).mean()), max_len, int(labels.sum()),
+             100.0 * float(labels.mean()), label_counts(data.label3, split.num_labels))
+    return split
+
+
+def epoch_batches(lengths: np.ndarray, batch_size: int, seed: int, epoch: int) -> list[np.ndarray]:
+    """Length-bucketed batch order for one epoch; a pure function of (seed, epoch) for resume."""
+    rng = np.random.default_rng([seed, epoch])
+    perm = rng.permutation(len(lengths))
+    mega = MEGA_BATCH_FACTOR * batch_size
+    batches: list[np.ndarray] = []
+    for start in range(0, len(perm), mega):
+        chunk = perm[start:start + mega]
+        chunk = chunk[np.argsort(-lengths[chunk], kind="stable")]
+        batches.extend(chunk[j:j + batch_size] for j in range(0, len(chunk), batch_size))
+    order = rng.permutation(len(batches))
+    return [batches[k] for k in order]
+
+
+def eval_batches(lengths: np.ndarray, batch_size: int) -> list[np.ndarray]:
+    order = np.argsort(-lengths, kind="stable")
+    return [order[i:i + batch_size] for i in range(0, len(order), batch_size)]
+
+
+class RowIndexDataset(Dataset):
+    """``torch.utils.data.Dataset`` over an ``EncodedSplit``: an item is a row index.
+
+    Membership and order come from the batch sampler (``epoch_batches`` / ``eval_batches``:
+    seeded shuffling + length bucketing), padding from the collate function, so every batch
+    the trainer sees goes through ``torch.utils.data.DataLoader`` exactly as the course
+    reference does, while the resume-exact batch order is kept.
+    """
+
+    def __init__(self, split: EncodedSplit) -> None:
+        self.split = split
+
+    def __len__(self) -> int:
+        return len(self.split)
+
+    def __getitem__(self, i: int) -> int:
+        return int(i)
+
+
+def make_loader(split: EncodedSplit, batches: list[np.ndarray], multiple: int,
+                micro_batch: int | None = None) -> DataLoader:
+    """DataLoader whose batch sampler is the precomputed batch list.
+
+    Yields ``(idx, ids, mask, labels, allowed)`` per batch, or with ``micro_batch`` set
+    ``(idx, [(ids, mask, labels, allowed), ...])`` where each micro-batch is padded to ITS
+    OWN longest row (the gradient-accumulation geometry the smoke test measured). Tensors
+    are produced on the CPU; the training loop moves them to the device. ``num_workers=0``
+    keeps the order a pure function of (seed, epoch) and avoids forking next to a live
+    CUDA context.
+    """
+    cpu = torch.device("cpu")
+
+    def collate(rows: list[int]):
+        idx = np.asarray(rows, dtype=np.int64)
+        if micro_batch is None:
+            return (idx, *split.collate(idx, multiple, cpu))
+        return idx, [split.collate(idx[c:c + micro_batch], multiple, cpu) for c in range(0, len(idx), micro_batch)]
+
+    # An explicit generator keeps the DataLoader from drawing its worker base seed from the
+    # global torch RNG at iteration start, which would shift the dropout stream and break
+    # bit-exact resume on the CPU dev path; the loader itself uses no randomness (num_workers=0,
+    # explicit batch_sampler), so the seed value is irrelevant.
+    return DataLoader(RowIndexDataset(split), batch_sampler=[[int(i) for i in b] for b in batches],
+                      collate_fn=collate, num_workers=0, generator=torch.Generator().manual_seed(0))
+
+
+def load_data(cfg: dict, args: TrainArgs, tok) -> tuple[EncodedSplit, EncodedSplit, int]:
+    processed = Path(cfg["paths"]["data_processed"])
+    holdout_source = str(cfg["data"]["holdout_test_source"])
+    max_len = int(cfg["model"]["max_seq_len"])
+    multiple = int(cfg["finetune"]["dynamic_padding_multiple"])
+
+    data = read_split(processed / TRAIN_FILE, holdout_source, args.limit_rows, args.seed, args.num_labels)
+    n_synth = 0
+    if args.synthetic == "on":
+        synth = processed / SYNTHETIC_FILE
+        if synth.exists():
+            # Synthetic rows carry their own label3/label_set (category "1:*" -> phishing, "0:*" -> legit).
+            sd = read_split(synth, holdout_source, int(cfg["data"]["synthetic"].get("max_rows", 2000)),
+                            args.seed, args.num_labels, force_source="synthetic")
+            n_synth = len(sd)
+            data = concat_labelled(data, sd)
+            LOG.info("synthetic supplement: %d rows appended to TRAIN only; classes %s", n_synth,
+                     label_counts(sd.label3, args.num_labels))
+        else:
+            LOG.warning("--synthetic on but %s is missing; training without synthetic rows", synth)
+    train = encode_split("train", tok, data, max_len, multiple)
+    vd = read_split(processed / VAL_FILE, holdout_source, args.limit_rows, args.seed, args.num_labels)
+    if args.limit_rows is not None:
+        LOG.warning("--limit-rows %d: validation is a seeded SUBSAMPLE too (dev only; metrics are not comparable)",
+                    args.limit_rows)
+    val = encode_split("val", tok, vd, max_len, multiple)
+    assert "synthetic" not in set(val.sources), "synthetic rows must never reach validation"
+    if args.num_labels == 3 and args.selection_metric == "val_macro_f1_3way" and not (val.label3 >= 0).any():
+        raise ValueError("selection metric val_macro_f1_3way but finetune_val has no row with label3 in {0,1,2}")
+    return train, val, n_synth
+
+
+def check_collate_matches_encode_batch(split: EncodedSplit, tok, idx: np.ndarray, max_len: int, multiple: int) -> None:
+    """The trained-on tensors must equal what ``encode_batch`` yields for the same texts."""
+    ids, mask, _, _ = split.collate(idx, multiple, torch.device("cpu"))
+    ref_ids, ref_mask = encode_batch(tok, [split.texts[int(i)] for i in idx], max_len, pad_to_multiple_of=multiple)
+    if ids.shape != ref_ids.shape or not torch.equal(ids, ref_ids) or not torch.equal(mask, ref_mask):
+        raise AssertionError("collated batch differs from tokenizer.encode_batch output")
+    LOG.info("collate == encode_batch on the first batch (%s): OK", tuple(ids.shape))
+
+
+# ---------------------------------------------------------------------------------- model
+def newest_pretrain_final(cfg: dict) -> Path | None:
+    root = Path(cfg["paths"]["checkpoints"]) / "pretrain"
+    finals = [p for p in root.glob("*/final") if (p / "model.safetensors").exists()] if root.exists() else []
+    return max(finals, key=lambda p: p.stat().st_mtime) if finals else None
+
+
+def read_pretrained(path: Path) -> tuple[dict[str, Tensor], dict[str, Any]]:
+    """state_dict + EncoderConfig dict from a Phase 1 final/ or step-NNNNNNN/ directory."""
+    weights = path / "model.safetensors"
+    if not weights.exists():
+        raise FileNotFoundError(f"{weights} not found")
+    cfg_json = path / "config.json"
+    if cfg_json.exists():
+        meta = common.read_json(cfg_json)
+    else:
+        with safe_open(str(weights), "pt") as fh:
+            raw = (fh.metadata() or {}).get("config")
+        if raw is None:
+            raise RuntimeError(f"{path}: no config.json and no 'config' safetensors metadata")
+        meta = json.loads(raw)
+    model_cfg = meta["model"] if isinstance(meta, dict) and "model" in meta else meta
+    return load_file(str(weights), device="cpu"), dict(model_cfg)
+
+
+def slice_layers(state: dict[str, Tensor], n_layers: int) -> dict[str, Tensor]:
+    """Keep encoder.layers.0..n_layers-1 (the instructor's 'number of layers' dial on a pretrained stack)."""
+    out: dict[str, Tensor] = {}
+    dropped = set()
+    for k, v in state.items():
+        if k.startswith("encoder.layers."):
+            i = int(k.split(".")[2])
+            if i >= n_layers:
+                dropped.add(i)
+                continue
+        out[k] = v
+    if dropped:
+        LOG.info("--n-layers %d: dropped pretrained layers %d..%d", n_layers, min(dropped), max(dropped))
+    return out
+
+
+def build_model(cfg: dict, args: TrainArgs, device: torch.device) -> tuple[EncoderForClassification, str]:
+    """Returns (model, init_descriptor) where init_descriptor is 'scratch' or 'pretrained:<path>'."""
+    # The head size comes from finetune.num_labels, never from config model: (the pretraining snapshot).
+    mcfg_dict = {**cfg["model"], "num_labels": args.num_labels}
+    if args.init == "scratch":
+        mcfg_dict["n_layers"] = args.n_layers
+        ecfg = EncoderConfig.from_dict(mcfg_dict)
+        common.set_seed(args.seed)
+        with torch.device(device):
+            model = EncoderForClassification(ecfg)
+        return model, "scratch"
+
+    path = Path(args.init.split(":", 1)[1]) if ":" in args.init else newest_pretrain_final(cfg)
+    if path is None:
+        raise FileNotFoundError("no checkpoints/pretrain/*/final found; pass --init pretrained:<dir> or --init scratch")
+    if not path.is_absolute():
+        path = common.project_root() / path
+    state, pre_cfg = read_pretrained(path)
+    if int(pre_cfg["n_layers"]) < args.n_layers:
+        raise ValueError(f"--n-layers {args.n_layers} > pretrained n_layers {pre_cfg['n_layers']}")
+    # Architecture from the checkpoint; regularisation from config.yaml; depth from the dial.
+    merged = {**pre_cfg, "n_layers": args.n_layers, "num_labels": args.num_labels,
+              "dropout": mcfg_dict["dropout"], "attention_dropout": mcfg_dict["attention_dropout"],
+              "grad_checkpointing": mcfg_dict.get("grad_checkpointing", False), "tie_mlm_head": pre_cfg.get("tie_mlm_head", True)}
+    ecfg = EncoderConfig.from_dict(merged)
+    common.set_seed(args.seed)
+    with torch.device(device):
+        model = EncoderForClassification.from_pretrained_encoder(slice_layers(state, args.n_layers), ecfg)
+    LOG.info("loaded pretrained encoder from %s (%d of %d layers)", path, args.n_layers, pre_cfg["n_layers"])
+    return model, f"pretrained:{path}"
+
+
+def freeze(model: EncoderForClassification, k: int) -> None:
+    if k <= 0:
+        return
+    for p in model.encoder.embeddings.parameters():
+        p.requires_grad_(False)
+    for layer in list(model.encoder.layers)[:k]:
+        for p in layer.parameters():
+            p.requires_grad_(False)
+    LOG.info("frozen: embeddings + encoder layers 0..%d", k - 1)
+
+
+# ----------------------------------------------------------------------------------- loss
+def class_weights(label3: np.ndarray, num_labels: int, device: torch.device) -> Tensor:
+    """Inverse-frequency class weights ``w_c = N / (C * n_c)`` over the FULLY labelled rows
+    (label3 >= 0) of the train split actually used. Unwanted rows have no class of their own;
+    they receive mean(w_phish, w_spam) through ``row_weights`` at loss time."""
+    full = label3[label3 >= 0]
+    counts = np.bincount(full, minlength=num_labels).astype(np.float64)
+    w = full.shape[0] / (float(num_labels) * np.maximum(counts, 1))
+    return torch.tensor(w, dtype=torch.float32, device=device)
+
+
+def partial_label_nll(logits: Tensor, allowed: Tensor) -> Tensor:
+    """Per-row ``-log sum_{c in allowed} softmax(logits)_c`` = logsumexp(all) - logsumexp(allowed).
+
+    One allowed class -> exactly ``F.cross_entropy(reduction="none")``; {phish, spam} ->
+    ``-log(p1 + p2)``. Computed in fp32 from bf16-autocast logits.
+    """
+    logits = logits.float()
+    if allowed.shape != logits.shape:
+        raise ValueError(f"allowed mask {tuple(allowed.shape)} does not match logits {tuple(logits.shape)}")
+    lse_all = torch.logsumexp(logits, dim=-1)
+    lse_allowed = torch.logsumexp(logits.masked_fill(~allowed, float("-inf")), dim=-1)
+    return lse_all - lse_allowed
+
+
+def row_weights(allowed: Tensor, weights: Tensor) -> Tensor:
+    """Mean of the class weights over each row's allowed classes: w[c] for a full label,
+    mean(w1, w2) for an "unwanted" row."""
+    a = allowed.to(weights.dtype)
+    return (a @ weights) / a.sum(dim=-1)
+
+
+def compute_loss(logits: Tensor, allowed: Tensor, kind: str, weights: Tensor | None, gamma: float) -> Tensor:
+    """The three loss dials, all defined over the partial-label NLL (see module docstring).
+
+    cross_entropy          mean(nll)
+    weighted_cross_entropy sum(w_r nll_r) / sum(w_r)   (F.cross_entropy(weight=)'s normalisation)
+    focal                  mean((1 - exp(-nll))^gamma * nll)   (= -(1 - p_t)^gamma log p_t; gamma 0 = plain)
+    """
+    nll = partial_label_nll(logits, allowed)
+    if kind == "cross_entropy":
+        return nll.mean()
+    if kind == "weighted_cross_entropy":
+        if weights is None:
+            raise ValueError("weighted_cross_entropy needs class weights")
+        w = row_weights(allowed, weights)
+        return (w * nll).sum() / w.sum()
+    if kind == "focal":
+        return ((1.0 - torch.exp(-nll)).pow(gamma) * nll).mean()
+    raise ValueError(f"unknown loss {kind!r}")
+
+
+# -------------------------------------------------------------------------------- metrics
+def classification_metrics(labels: np.ndarray, probs: np.ndarray, threshold: float = DECISION_THRESHOLD) -> dict[str, Any]:
+    from sklearn.metrics import (accuracy_score, average_precision_score, confusion_matrix,
+                                 precision_recall_curve, precision_recall_fscore_support, roc_auc_score)
+
+    pred = (probs >= threshold).astype(np.int64)
+    p, r, f1, _ = precision_recall_fscore_support(labels, pred, average="binary", pos_label=1, zero_division=0)
+    cm = confusion_matrix(labels, pred, labels=[0, 1])
+    try:
+        roc = float(roc_auc_score(labels, probs))
+        pr_auc = float(average_precision_score(labels, probs))
+        pc, rc, th = precision_recall_curve(labels, probs)
+        f1s = 2 * pc[:-1] * rc[:-1] / np.maximum(pc[:-1] + rc[:-1], 1e-12)
+        j = int(np.argmax(f1s))
+        opt_t, opt_f1 = float(th[j]), float(f1s[j])
+    except ValueError:  # a single class present (tiny dev subsets)
+        roc = pr_auc = opt_t = opt_f1 = float("nan")
+    return {
+        "accuracy": float(accuracy_score(labels, pred)), "precision": float(p), "recall": float(r), "f1": float(f1),
+        "roc_auc": roc, "pr_auc": pr_auc,
+        "tn": int(cm[0, 0]), "fp": int(cm[0, 1]), "fn": int(cm[1, 0]), "tp": int(cm[1, 1]),
+        "threshold": threshold, "f1_optimal_threshold": opt_t, "f1_at_optimal_threshold": opt_f1,
+    }
+
+
+def three_way_metrics(label3: np.ndarray, probs: np.ndarray) -> dict[str, Any]:
+    """Metric set (b) on the rows with a full three-way label (label3 in {0, 1, 2}).
+
+    ``probs`` is ``[n, 3]``. Prediction = argmax over the three classes. Keys (flat, so they
+    drop straight into epochs.csv): n_3way, accuracy_3way, macro_f1_3way (macro over ALL three
+    classes, a class absent from the slice scores 0, so tiny dev subsets read low),
+    {precision,recall,f1}_{legitimate,phishing,spam}, cm3_<true><pred> with l/p/s initials
+    (cm3_ps = true phishing predicted spam), and the phishing-vs-spam confusion restricted to
+    true unwanted rows where the prediction is the argmax over {phishing, spam} only
+    ("given it is unwanted, which kind?"): n_unwanted_3way, pvs_phish_as_phish,
+    pvs_phish_as_spam, pvs_spam_as_phish, pvs_spam_as_spam.
+    Returns None/0 placeholders (same keys) when the slice is empty or the model is binary: None,
+    not NaN, so best/config.json and experiments.jsonl of a binary run stay strict JSON.
+    """
+    from sklearn.metrics import confusion_matrix, precision_recall_fscore_support
+
+    out: dict[str, Any] = {k: (0 if k.startswith(("n_", "cm3_", "pvs_")) else None) for k in THREE_WAY_KEYS}
+    sel = label3 >= 0
+    if probs.ndim != 2 or probs.shape[1] != 3 or not sel.any():
+        return out
+    y, p = label3[sel], probs[sel]
+    pred = p.argmax(axis=1)
+    cm = confusion_matrix(y, pred, labels=[0, 1, 2])
+    prec, rec, f1, _ = precision_recall_fscore_support(y, pred, labels=[0, 1, 2], zero_division=0)
+    out.update(n_3way=int(sel.sum()), accuracy_3way=float((pred == y).mean()), macro_f1_3way=float(f1.mean()))
+    for c, slug in enumerate(CLASS_SLUGS):
+        out[f"precision_{slug}"], out[f"recall_{slug}"], out[f"f1_{slug}"] = float(prec[c]), float(rec[c]), float(f1[c])
+    for i in range(3):
+        for j in range(3):
+            out[f"cm3_{CLASS_SLUGS[i][0]}{CLASS_SLUGS[j][0]}"] = int(cm[i, j])
+    unw = y >= 1
+    if unw.any():
+        kind = 1 + p[unw][:, 1:3].argmax(axis=1)
+        cm2 = confusion_matrix(y[unw], kind, labels=[1, 2])
+        out.update(n_unwanted_3way=int(unw.sum()), pvs_phish_as_phish=int(cm2[0, 0]), pvs_phish_as_spam=int(cm2[0, 1]),
+                   pvs_spam_as_phish=int(cm2[1, 0]), pvs_spam_as_spam=int(cm2[1, 1]))
+    return out
+
+
+def metric_sets(res: dict[str, Any]) -> dict[str, Any]:
+    """Structured view of a flat evaluation result for best/config.json and experiments.jsonl:
+    ``binary_unwanted`` (metric set a) and ``three_way`` (metric set b, None for a binary model
+    or an empty slice)."""
+    binary = {k: res.get(k) for k in (*METRIC_KEYS, "tn", "fp", "fn", "tp", "threshold", "f1_optimal_threshold",
+                                      "f1_at_optimal_threshold", "n")}
+    binary["positive_class"] = UNWANTED_NAME
+    three = None
+    if res.get("n_3way"):
+        three = {
+            "n": res["n_3way"], "accuracy": res["accuracy_3way"], "macro_f1": res["macro_f1_3way"],
+            "per_class": {LABEL_NAMES[c]: {m: res[f"{m}_{slug}"] for m in ("precision", "recall", "f1")}
+                          for c, slug in enumerate(CLASS_SLUGS)},
+            "confusion": {"order": [LABEL_NAMES[c] for c in range(3)], "rows_true_cols_pred":
+                          [[res[f"cm3_{a[0]}{b[0]}"] for b in CLASS_SLUGS] for a in CLASS_SLUGS]},
+            "phish_vs_spam_on_true_unwanted": {
+                "n": res["n_unwanted_3way"], "order": [LABEL_NAMES[1], LABEL_NAMES[2]],
+                "rows_true_cols_pred": [[res["pvs_phish_as_phish"], res["pvs_phish_as_spam"]],
+                                        [res["pvs_spam_as_phish"], res["pvs_spam_as_spam"]]]},
+        }
+    return {"binary_unwanted": binary, "three_way": three}
+
+
+def autocast_ctx(device: torch.device):
+    """bf16 autocast on CUDA; a no-op on the CPU dev path (fp32 end to end)."""
+    if device.type == "cuda":
+        return torch.autocast(device_type="cuda", dtype=torch.bfloat16)
+    return contextlib.nullcontext()
+
+
+def unwanted_prob(probs: Tensor) -> Tensor:
+    """P(unwanted) = p_phish + p_spam (3 classes) or p_1 (binary)."""
+    return probs[:, 1:].sum(dim=-1)
+
+
+@torch.inference_mode()
+def evaluate(model: EncoderForClassification, split: EncodedSplit, args: TrainArgs, multiple: int,
+             device: torch.device) -> dict[str, Any]:
+    """Both metric sets on ``split``. ``val_loss`` is the unweighted partial-label NLL (the
+    plain cross_entropy dial) so it is comparable across loss variants."""
+    model.eval()
+    t0 = common.timer()
+    probs, labels, label3s, loss_sum = [], [], [], 0.0
+    loader = make_loader(split, eval_batches(split.lengths, args.eval_batch_size), multiple)
+    for idx, ids, mask, y, allowed in loader:
+        ids, mask, y, allowed = (t.to(device, non_blocking=True) for t in (ids, mask, y, allowed))
+        with autocast_ctx(device):
+            logits = model(ids, mask)
+        logits = logits.float()
+        loss_sum += float(partial_label_nll(logits, allowed).sum())
+        probs.append(logits.softmax(-1).cpu().numpy())
+        labels.append(y.cpu().numpy())
+        # eval_batches() visits rows in length order, so EVERY target must be gathered with the same idx
+        # as the predictions; split.label3 in storage order would score a permutation of the model.
+        label3s.append(split.label3[idx])
+    model.train()
+    probs_np, labels_np, label3_np = np.concatenate(probs), np.concatenate(labels), np.concatenate(label3s)
+    out = classification_metrics(labels_np, unwanted_prob(torch.from_numpy(probs_np)).numpy())
+    out["f1_unwanted"] = out["f1"]
+    out.update(three_way_metrics(label3_np, probs_np))
+    out["val_loss"] = loss_sum / max(1, len(labels_np))
+    out["n"] = int(len(labels_np))
+    out["eval_seconds"] = round(common.timer() - t0, 1)
+    return out
+
+
+def selection_value(res: dict[str, Any], metric: str) -> float:
+    v = res.get(SELECTION_KEY[metric])
+    return float("-inf") if v is None or (isinstance(v, float) and math.isnan(v)) else float(v)
+
+
+# ------------------------------------------------------------------------------- run state
+@dataclass
+class RunState:
+    step: int = 0
+    epoch: int = 0
+    batch_in_epoch: int = 0
+    best_score: float = -1.0        # value of the selection metric at best/ (named best_f1 before entry 6)
+    best_step: int = 0
+    best_epoch: int = 0
+    evals_without_improvement: int = 0
+    train_seconds: float = 0.0
+    eval_seconds: float = 0.0
+    peak_reserved_gb: float = 0.0
+    peak_allocated_gb: float = 0.0
+    best_val: dict[str, Any] | None = None
+    last_val: dict[str, Any] | None = None
+    n_train: int = 0
+    n_val: int = 0
+    n_synthetic: int = 0
+    init: str = ""
+    stopped_by: str = ""
+    resumed_from_step: int | None = None
+    metrics_tail: list[dict[str, Any]] = field(default_factory=list)
+
+
+class StopFlag:
+    """SIGINT/SIGTERM -> finish the current micro-step, save last/, exit 0. Second signal -> default."""
+
+    def __init__(self) -> None:
+        self.requested: str | None = None
+        self._old = {s: signal.signal(s, self._handle) for s in (signal.SIGINT, signal.SIGTERM)}
+
+    def _handle(self, signum: int, frame: Any) -> None:
+        if self.requested:
+            signal.signal(signum, self._old[signum])
+            raise KeyboardInterrupt
+        self.requested = signal.Signals(signum).name
+        LOG.warning("%s received: finishing the current step, saving last/ and exiting", self.requested)
+
+
+def rng_states(device: torch.device) -> dict[str, Any]:
+    """RNG snapshot for resume. The CUDA generator is read only when training ON CUDA:
+    ``torch.cuda.get_rng_state_all`` goes through ``_lazy_init`` and would create a CUDA context
+    on a GPU the --cpu dev path must leave alone (the production pretraining run lives there)."""
+    return {"python": random.getstate(), "numpy": np.random.get_state(), "torch_cpu": torch.get_rng_state(),
+            "torch_cuda": torch.cuda.get_rng_state_all() if device.type == "cuda" else []}
+
+
+def restore_rng(states: dict[str, Any], device: torch.device) -> None:
+    random.setstate(states["python"])
+    np.random.set_state(states["numpy"])
+    torch.set_rng_state(states["torch_cpu"])
+    if device.type == "cuda" and states.get("torch_cuda"):
+        torch.cuda.set_rng_state_all(states["torch_cuda"])
+
+
+def json_clean(obj: Any) -> Any:
+    """Recursively replace non-finite floats (NaN/inf from a degenerate metric) with None so every
+    JSON artifact (best/config.json, last/config.json, experiments.jsonl) is strict JSON; the stdlib
+    encoder would otherwise emit the bare token ``NaN`` that non-Python parsers reject."""
+    if isinstance(obj, float):
+        return obj if math.isfinite(obj) else None
+    if isinstance(obj, np.floating):
+        return float(obj) if np.isfinite(obj) else None
+    if isinstance(obj, dict):
+        return {k: json_clean(v) for k, v in obj.items()}
+    if isinstance(obj, (list, tuple)):
+        return [json_clean(v) for v in obj]
+    return obj
+
+
+def save_model_dir(model: EncoderForClassification, out: Path, step: int, extra: dict[str, Any]) -> None:
+    out.mkdir(parents=True, exist_ok=True)
+    sd = {k: v.detach().to("cpu").contiguous() for k, v in model.state_dict().items()}
+    cfg_dict = model.config.to_dict()
+    save_file(sd, str(out / "model.safetensors"),
+              metadata={"format": "pt", "step": str(step), "config": json.dumps(cfg_dict)})
+    common.write_json(json_clean({"model": cfg_dict, "tokenizer": "tokenizer.json", **extra}), out / "config.json")
+
+
+def save_last(run_dir: Path, model: EncoderForClassification, opt: torch.optim.Optimizer,
+              sched: torch.optim.lr_scheduler.LRScheduler, state: RunState, args: TrainArgs,
+              schedule: dict[str, int], device: torch.device) -> None:
+    t0 = common.timer()
+    last = run_dir / "last"
+    save_model_dir(model, last, state.step, {"finetune": args.to_record(), "state": _state_record(state)})
+    torch.save({
+        "optimizer": opt.state_dict(), "scheduler": sched.state_dict(), "state": asdict(state),
+        "args": args.to_record(), "schedule": schedule, "rng": rng_states(device),
+    }, last / "trainer_state.pt")
+    LOG.info("saved last/ at step %d (%.1fs)", state.step, common.timer() - t0)
+
+
+def _state_record(state: RunState) -> dict[str, Any]:
+    d = asdict(state)
+    d.pop("metrics_tail", None)
+    return d
+
+
+def write_csv_row(path: Path, row: dict[str, Any], header: list[str]) -> None:
+    new = not path.exists()
+    with path.open("a", newline="", encoding="utf-8") as fh:
+        w = csv.DictWriter(fh, fieldnames=header, extrasaction="ignore")
+        if new:
+            w.writeheader()
+        w.writerow({k: ("" if row.get(k) is None else row.get(k)) for k in header})
+
+
+def trim_csv(path: Path, max_step: int) -> None:
+    """Drop rows with step > max_step: a crash between checkpoints leaves rows past the last/ being
+    resumed, and the resumed steps would otherwise be appended a second time."""
+    if not path.exists():
+        return
+    with path.open(newline="", encoding="utf-8") as fh:
+        reader = csv.DictReader(fh)
+        header, rows = list(reader.fieldnames or []), list(reader)
+    keep = [r for r in rows if r.get("step") and int(float(r["step"])) <= max_step]
+    if len(keep) == len(rows):
+        return
+    tmp = path.with_suffix(path.suffix + ".tmp")
+    with tmp.open("w", newline="", encoding="utf-8") as fh:
+        w = csv.DictWriter(fh, fieldnames=header)
+        w.writeheader()
+        w.writerows(keep)
+    os.replace(tmp, path)
+    LOG.info("%s: dropped %d row(s) logged after step %d (crash between checkpoints)", path.name, len(rows) - len(keep), max_step)
+
+
+METRICS_HEADER = ["step", "epoch", "loss", "lr", "examples_per_s", "max_alloc_gb", "max_reserved_gb", "elapsed_s",
+                  "eta_s", "val_loss", "val_f1", "val_roc_auc", "val_macro_f1_3way"]
+# Binary roll-up columns first (unchanged names, so old epochs.csv readers keep working), then the three-way set.
+EPOCHS_HEADER = ["epoch", "step", "is_epoch_end", "n", "val_loss", "accuracy", "precision", "recall", "f1", "roc_auc",
+                 "pr_auc", "tn", "fp", "fn", "tp", "threshold", "f1_optimal_threshold", "f1_at_optimal_threshold",
+                 "f1_unwanted", *THREE_WAY_KEYS, "eval_seconds", "is_best"]
+
+
+# ----------------------------------------------------------------------------- experiments
+def experiment_record(args: TrainArgs, state: RunState, params: dict[str, Any], steps_per_epoch: int) -> dict[str, Any]:
+    val = state.best_val or {}
+    return {
+        "run_name": args.run_name,
+        "timestamp": time.strftime("%Y-%m-%dT%H:%M:%S"),
+        "init": state.init,
+        "dials": {
+            "n_layers": args.n_layers, "frozen_layers": args.freeze_layers,
+            "epochs_planned": args.epochs, "epochs_run": round(state.step / max(1, steps_per_epoch), 3),
+            "loss": args.loss, "focal_gamma": args.focal_gamma if args.loss == "focal" else None,
+            "lr": args.lr, "batch_size": args.batch_size, "micro_batch": args.micro_batch,
+            "class_weights": "inverse_frequency" if args.loss == "weighted_cross_entropy" else None,
+            "synthetic": args.synthetic, "synthetic_rows": state.n_synthetic,
+            "warmup_fraction": args.warmup_fraction, "max_steps": args.max_steps, "limit_rows": args.limit_rows,
+            "seed": args.seed, "schedule": args.schedule, "selection_metric": args.selection_metric,
+            "pooling": args.pooling, "num_labels": args.num_labels, "device": "cpu" if args.cpu else "cuda",
+        },
+        "params": params,
+        "steps_run": state.step,
+        "n_train": state.n_train, "n_val": state.n_val,
+        "train_seconds": round(state.train_seconds, 1),
+        "peak_vram_gb": round(state.peak_reserved_gb, 3),
+        "best_epoch": state.best_epoch, "best_step": state.best_step,
+        "best_score": state.best_score if val else None,
+        # Old binary keys kept verbatim (finetune.py list / evaluate.py read them); the three-way set is added.
+        "val": ({**{k: val.get(k) for k in METRIC_KEYS}, "f1_unwanted": val.get("f1"),
+                 "macro_f1_3way": val.get("macro_f1_3way"),
+                 "per_class": {LABEL_NAMES[c]: {m: val.get(f"{m}_{slug}") for m in ("precision", "recall", "f1")}
+                               for c, slug in enumerate(CLASS_SLUGS)} if val.get("n_3way") else None,
+                 "three_way": metric_sets(val)["three_way"]} if val else None),
+        "val_threshold": DECISION_THRESHOLD,
+        "label_names": [LABEL_NAMES[c] for c in range(args.num_labels)] if args.num_labels == 3 else [LABEL_NAMES[0], UNWANTED_NAME],
+        "stopped_by": state.stopped_by,
+        "resumed_from_step": state.resumed_from_step,
+        "test": None,
+    }
+
+
+def upsert_experiment(path: Path, rec: dict[str, Any]) -> str:
+    """ONE train line per run_name: a resumed invocation replaces its earlier train line instead
+    of appending a duplicate; evaluate.py's lines (test != null) are never touched."""
+    if rec.get("test") is not None:
+        raise ValueError("upsert_experiment writes train lines only (test must be None); a test record would "
+                         "overwrite the run's train line")
+    lines = path.read_text(encoding="utf-8").splitlines() if path.exists() else []
+    out, replaced = [], False
+    rec = json_clean(rec)   # strict JSON: no bare NaN tokens for binary runs' three-way placeholders
+    for line in lines:
+        if not line.strip():
+            continue
+        try:
+            old = json.loads(line)
+        except json.JSONDecodeError:
+            out.append(line)
+            continue
+        if not replaced and old.get("run_name") == rec["run_name"] and old.get("test") is None:
+            out.append(json.dumps(rec, allow_nan=False))
+            replaced = True
+        else:
+            out.append(line)
+    if not replaced:
+        out.append(json.dumps(rec, allow_nan=False))
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_suffix(".jsonl.tmp")
+    tmp.write_text("\n".join(out) + "\n", encoding="utf-8")
+    os.replace(tmp, path)
+    return "replaced" if replaced else "appended"
+
+
+def run_log_block(args: TrainArgs, state: RunState, run_dir: Path, params: dict[str, Any]) -> str:
+    v = state.best_val or {}
+    nan = float("nan")
+    three = ""
+    if v.get("n_3way"):
+        three = (f" Three-way on {v['n_3way']:,} fully-labelled val rows: macro-F1 {v['macro_f1_3way']:.4f}, accuracy "
+                 f"{v['accuracy_3way']:.4f}; per-class F1 " +
+                 ", ".join(f"{LABEL_NAMES[c]} {v[f'f1_{s}']:.4f}" for c, s in enumerate(CLASS_SLUGS)) +
+                 f"; phishing-vs-spam on true unwanted (n={v['n_unwanted_3way']:,}): phish->phish {v['pvs_phish_as_phish']}, "
+                 f"phish->spam {v['pvs_phish_as_spam']}, spam->phish {v['pvs_spam_as_phish']}, spam->spam {v['pvs_spam_as_spam']}.")
+    return "\n".join([
+        f"## Fine-tune run `{args.run_name}` — {time.strftime('%Y-%m-%d %H:%M')}",
+        f"**Dials.** init {state.init}; {args.num_labels} classes; {args.n_layers} layers, {args.freeze_layers} frozen; loss {args.loss}"
+        f"{f' (gamma {args.focal_gamma})' if args.loss == 'focal' else ''}; lr {args.lr:g}; batch {args.batch_size} "
+        f"(micro {args.micro_batch}); epochs {args.epochs} planned; synthetic {args.synthetic} ({state.n_synthetic} rows); "
+        f"selection {args.selection_metric}; train rows {state.n_train:,}, val rows {state.n_val:,}; params {params['total']:,} "
+        f"({params['trainable']:,} trainable).",
+        f"**Result.** {state.step} steps, stopped by {state.stopped_by}; training time {state.train_seconds / 60:.1f} min "
+        f"(eval {state.eval_seconds / 60:.1f} min of it); peak reserved {state.peak_reserved_gb:.2f} GiB. Best val at step "
+        f"{state.best_step} (epoch {state.best_epoch}), {UNWANTED_NAME} roll-up: F1 {v.get('f1', nan):.4f}, precision "
+        f"{v.get('precision', nan):.4f}, recall {v.get('recall', nan):.4f}, accuracy {v.get('accuracy', nan):.4f}, ROC-AUC "
+        f"{v.get('roc_auc', nan):.4f}, PR-AUC {v.get('pr_auc', nan):.4f}; F1-optimal threshold {v.get('f1_optimal_threshold', nan):.3f}."
+        f"{three} Checkpoints: {run_dir}/best, {run_dir}/last.",
+    ])
+
+
+# ------------------------------------------------------------------------------- training
+CPU_VRAM = {"allocated_gb": 0.0, "reserved_gb": 0.0, "max_allocated_gb": 0.0, "max_reserved_gb": 0.0}
+
+
+def backfill_legacy_args(saved: dict[str, Any], run_dir: Path) -> None:
+    """A trainer_state.pt written before RUN_LOG entry 6 has no ``num_labels`` / ``selection_metric``
+    in its args. Fill them from what that run actually was (the head size recorded in
+    last/config.json model.num_labels; the only selection metric that existed, val_f1_phishing) so the
+    saved-overrides-config rule resolves the run as the binary run it is, instead of the current config's
+    3-class head crashing in load_state_dict with a size mismatch."""
+    args = saved.get("args") or {}
+    if "num_labels" not in args:
+        try:
+            n = int(common.read_json(run_dir / "last" / "config.json")["model"]["num_labels"])
+        except (OSError, KeyError, TypeError, ValueError, json.JSONDecodeError):
+            n = 2
+        args["num_labels"] = n
+        LOG.info("--resume: legacy trainer_state.pt without num_labels; using %d from last/config.json", n)
+    if "selection_metric" not in args:
+        args["selection_metric"] = "val_f1_phishing"
+    saved["args"] = args
+
+
+def lr_lambda_factory(total_steps: int, warmup_steps: int):
+    def lr_lambda(completed: int) -> float:
+        if completed < warmup_steps:
+            return (completed + 1) / warmup_steps
+        return max(0.0, (total_steps - completed) / max(1, total_steps - warmup_steps))
+    return lr_lambda
+
+
+def train(ns: argparse.Namespace, cfg: dict) -> int:
+    t_invocation = common.timer()
+    # Decide the device FIRST and touch torch.cuda only on the CUDA branch. Under --cpu the GPU may be
+    # (and today is) occupied by the production pretraining run: torch.cuda.current_device() /
+    # mem_get_info() / get_rng_state_all() all run _lazy_init(), which creates a CUDA context there.
+    # Hiding the devices from this process is the belt to that braces: nothing has called CUDA yet.
+    if ns.cpu:
+        # Hide the devices before the CUDA runtime is first queried (torch import does not query it): then
+        # torch.cuda.is_available() is False and torch's own AdamW.step() health check, which otherwise calls
+        # torch.accelerator.current_stream() -> _lazy_init() on any visible GPU, is skipped. If a GPU is still
+        # visible the contract cannot be honoured from inside this process, so refuse rather than risk it.
+        os.environ["CUDA_VISIBLE_DEVICES"] = ""
+        if torch.cuda.is_initialized() or torch.cuda.is_available():
+            LOG.error("--cpu: a CUDA device is still visible to this process (CUDA was queried before the devices could "
+                      "be hidden); run with CUDA_VISIBLE_DEVICES=\"\" in the environment so the occupied GPU is never touched")
+            return 2
+        device = torch.device("cpu")
+        device_info = {"device": "cpu", "torch": torch.__version__}
+        LOG.warning("--cpu: dev path on the CPU with the config as given (no autocast, no VRAM guard, CUDA never "
+                    "initialised; CUDA_VISIBLE_DEVICES cleared for this process); not for real runs")
+    elif not torch.cuda.is_available():
+        LOG.error("CUDA is required for fine-tuning (--cpu is the dev-only escape hatch)")
+        return 2
+    else:
+        device = torch.device("cuda")
+        device_info = common.print_device_info()
+        LOG.info("PYTORCH_CUDA_ALLOC_CONF=%s", os.environ.get("PYTORCH_CUDA_ALLOC_CONF"))
+
+    run_dir = Path(cfg["paths"]["checkpoints"]) / "finetune" / ns.run_name
+    last_state_path = run_dir / "last" / "trainer_state.pt"
+    saved: dict[str, Any] | None = None
+    if ns.resume:
+        if not last_state_path.exists():
+            LOG.error("--resume: %s not found", last_state_path)
+            return 2
+        # Own file (optimizer tensors + Python/NumPy RNG tuples), hence weights_only=False.
+        saved = torch.load(last_state_path, map_location="cpu", weights_only=False)
+        backfill_legacy_args(saved, run_dir)
+    elif run_dir.exists() and any(run_dir.iterdir()):
+        if not ns.overwrite:
+            LOG.error("%s exists; use --resume to continue it, --overwrite to discard it, or a new --run-name", run_dir)
+            return 2
+        import shutil
+        shutil.rmtree(run_dir)
+        LOG.warning("--overwrite: removed %s", run_dir)
+
+    try:
+        args = TrainArgs.resolve(ns, cfg, saved["args"] if saved else None)
+    except ValueError as e:
+        LOG.error("dial error: %s", e)
+        return 2
+    args.run_name = ns.run_name
+    args.cpu = bool(ns.cpu)   # the device is this invocation's choice, never restored from last/
+    if saved:
+        changed = {k: (saved["args"].get(k), getattr(ns, k)) for k in RESUME_FIXED_DIALS
+                   if getattr(ns, k, None) is not None and getattr(ns, k) != saved["args"].get(k)}
+        if changed:
+            LOG.error("--resume: %s cannot change on resume (saved -> requested); the restored optimizer state and "
+                      "batch order would not match. Start a new --run-name instead", changed)
+            return 2
+    LOG.info("resolved dials: %s", args.to_record())
+    common.set_seed(args.seed)
+    vr = cfg["vram"]
+    max_len = int(cfg["model"]["max_seq_len"])
+    multiple = int(cfg["finetune"]["dynamic_padding_multiple"])
+
+    tok = load_tokenizer(cfg["paths"]["tokenizer"])
+    train_split, val_split, n_synth = load_data(cfg, args, tok)
+    check_collate_matches_encode_batch(train_split, tok, epoch_batches(train_split.lengths, args.batch_size, args.seed, 0)[0],
+                                       max_len, multiple)
+    if saved and (saved["state"]["n_train"] != len(train_split) or saved["state"]["n_val"] != len(val_split)):
+        LOG.error("--resume: data changed (train %d -> %d, val %d -> %d); the batch order would not line up",
+                  saved["state"]["n_train"], len(train_split), saved["state"]["n_val"], len(val_split))
+        return 2
+
+    try:
+        model, init_desc = build_model(cfg, args, device)
+    except (FileNotFoundError, ValueError, RuntimeError) as e:
+        LOG.error("model init failed: %s", e)
+        return 2
+    if saved:
+        sd = load_file(str(run_dir / "last" / "model.safetensors"), device="cpu")
+        try:
+            model.load_state_dict(sd, strict=True)
+        except RuntimeError as e:   # head / depth mismatch that slipped past the dial checks
+            LOG.error("--resume: last/model.safetensors does not fit the resolved dials (num_labels %d, n_layers %d): %s",
+                      args.num_labels, args.n_layers, str(e).splitlines()[0])
+            return 2
+    freeze(model, args.freeze_layers)
+    model.train()
+    params = count_parameters(model)
+    LOG.info("EncoderForClassification params: %s", params)
+    if tok.get_vocab_size() > model.config.vocab_size:
+        raise AssertionError("tokenizer vocab larger than model vocab")
+
+    steps_per_epoch = math.ceil(len(train_split) / args.batch_size)
+    total_steps = steps_per_epoch * args.epochs
+    if args.max_steps is not None:
+        total_steps = min(total_steps, args.max_steps)
+    warmup_steps = max(1, math.ceil(args.warmup_fraction * total_steps))
+    schedule = {"steps_per_epoch": steps_per_epoch, "total_steps": total_steps, "warmup_steps": warmup_steps}
+    LOG.info("schedule: %d steps/epoch x %d epochs -> %d optimizer steps (max_steps=%s), warmup %d, "
+             "micro-batch %d x %d accumulation", steps_per_epoch, args.epochs, total_steps, args.max_steps, warmup_steps,
+             args.micro_batch, args.batch_size // args.micro_batch)
+
+    opt_cfg = {"weight_decay": args.weight_decay, "betas": list(args.betas), "eps": args.eps}
+    opt = build_optimizer(model, args.lr, opt_cfg)
+    lr_lambda = lr_lambda_factory(total_steps, warmup_steps)
+    sched = torch.optim.lr_scheduler.LambdaLR(opt, lr_lambda)
+    weights = class_weights(train_split.label3, args.num_labels, device) if args.loss == "weighted_cross_entropy" else None
+    if weights is not None:
+        names = [LABEL_NAMES[c] for c in range(3)] if args.num_labels == 3 else [LABEL_NAMES[0], UNWANTED_NAME]
+        LOG.info("inverse-frequency class weights %s from fully-labelled counts %s%s", dict(zip(names, [round(float(w), 4) for w in weights])),
+                 np.bincount(train_split.label3[train_split.label3 >= 0], minlength=args.num_labels).tolist(),
+                 f"; {int((train_split.label3 < 0).sum())} unwanted rows weighted mean(w_phish, w_spam)={float(weights[1:].mean()):.4f}"
+                 if args.num_labels == 3 else "")
+    if args.num_labels == 3:
+        LOG.info("label contract: classes %s; binary roll-up '%s' = P(%s) + P(%s); selection metric %s",
+                 LABEL_NAMES, UNWANTED_NAME, LABEL_NAMES[1], LABEL_NAMES[2], args.selection_metric)
+
+    state = RunState(n_train=len(train_split), n_val=len(val_split), n_synthetic=n_synth, init=init_desc)
+    if saved:
+        st = dict(saved["state"])
+        if "best_f1" in st:   # trainer_state.pt written before entry 6
+            st["best_score"] = st.pop("best_f1")
+        known = {f.name for f in fields(RunState)}
+        state = RunState(**{**{k: v for k, v in st.items() if k in known}, "stopped_by": "", "init": st["init"]})
+        state.resumed_from_step = state.step
+        opt.load_state_dict(saved["optimizer"])
+        sched.load_state_dict(saved["scheduler"])
+        if args.lr != float(saved["args"]["lr"]):
+            # load_state_dict restored the saved base lr; re-apply the override so the next optimizer step
+            # (which runs before the next sched.step()) and every later one use the lr the record claims.
+            scale = lr_lambda(sched.last_epoch)
+            for g in opt.param_groups:
+                g["initial_lr"] = args.lr
+                g["lr"] = args.lr * scale
+            sched.base_lrs = [args.lr] * len(opt.param_groups)
+            LOG.warning("--lr %g overrides the saved %g from step %d onward (steps 1..%d used the old lr); current lr %.3e",
+                        args.lr, float(saved["args"]["lr"]), state.step, state.step, args.lr * scale)
+        if saved["schedule"] != schedule:
+            LOG.warning("schedule changed on resume %s -> %s; the LR lambda now follows the new totals from step %d",
+                        saved["schedule"], schedule, state.step)
+        restore_rng(saved["rng"], device)
+        LOG.info("resumed %s at step %d (epoch %d, batch %d/%d); best %s %.4f at step %d; %.1f s trained so far",
+                 ns.run_name, state.step, state.epoch, state.batch_in_epoch, steps_per_epoch, args.selection_metric,
+                 state.best_score, state.best_step, state.train_seconds)
+        if args.max_steps is not None and state.step >= args.max_steps:
+            LOG.warning("nothing to resume: step %d already reached --max-steps %d; raise --max-steps or --epochs to continue",
+                        state.step, args.max_steps)
+            return 0
+        if state.epoch >= args.epochs:
+            LOG.warning("nothing to resume: epoch %d already reached --epochs %d; raise --epochs to continue",
+                        state.epoch, args.epochs)
+            return 0
+        if state.evals_without_improvement >= args.patience:
+            LOG.warning("resuming after early stopping (%d/%d evaluations without improvement): the run stops again at the "
+                        "next evaluation unless %s improves", state.evals_without_improvement, args.patience, args.selection_metric)
+
+    run_dir.mkdir(parents=True, exist_ok=True)
+    metrics_csv, epochs_csv = run_dir / "metrics.csv", run_dir / "epochs.csv"
+    if saved:
+        trim_csv(metrics_csv, state.step)
+        trim_csv(epochs_csv, state.step)
+    stop = StopFlag()
+    if device.type == "cuda":
+        torch.cuda.reset_peak_memory_stats()
+    band_warned = False
+    label_names = [LABEL_NAMES[c] for c in range(3)] if args.num_labels == 3 else [LABEL_NAMES[0], UNWANTED_NAME]
+
+    def record_eval(is_epoch_end: bool) -> None:
+        nonlocal state
+        res = evaluate(model, val_split, args, multiple, device)
+        state.eval_seconds += res["eval_seconds"]
+        state.last_val = res
+        score = selection_value(res, args.selection_metric)
+        improved = score > state.best_score
+        if improved:
+            state.best_score, state.best_step, state.best_epoch, state.best_val = score, state.step, state.epoch, res
+            state.evals_without_improvement = 0
+            save_model_dir(model, run_dir / "best", state.step, {
+                "finetune": args.to_record(), "best_epoch": state.epoch, "best_step": state.step,
+                "num_labels": args.num_labels, "label_names": label_names, "binary_rollup_name": UNWANTED_NAME,
+                "selection_metric": args.selection_metric, "selection_value": score,
+                "val_metrics": res, "val_metric_sets": metric_sets(res),
+                "thresholds": {"decision": DECISION_THRESHOLD, "decision_applies_to": "P(unwanted) = sum of non-legitimate probabilities",
+                               "val_f1_optimal": res["f1_optimal_threshold"], "val_f1_at_optimal": res["f1_at_optimal_threshold"]},
+                "init": state.init, "params": params,
+            })
+        else:
+            state.evals_without_improvement += 1
+        three = ""
+        if res.get("n_3way"):
+            three = (f" | 3-way n={res['n_3way']} macro-F1 {res['macro_f1_3way']:.4f} acc {res['accuracy_3way']:.4f} F1[L P S]=[%s] "
+                     f"cm3[rows true L,P,S; cols pred]=[%s] phish/spam on unwanted [pp ps sp ss]=[%d %d %d %d]" % (
+                         " ".join(f"{res[f'f1_{s}']:.3f}" for s in CLASS_SLUGS),
+                         " ".join(f"{res[k]}" for k in CM3_KEYS), *(res[k] for k in PVS_KEYS)))
+        LOG.info("eval step %d epoch %d%s: n=%d loss %.4f | unwanted roll-up acc %.4f P %.4f R %.4f F1 %.4f ROC-AUC %.4f "
+                 "PR-AUC %.4f cm[tn fp fn tp]=[%d %d %d %d] F1-opt thr %.3f (F1 %.4f)%s | %s=%.4f %s (%.1fs)",
+                 state.step, state.epoch, " [epoch end]" if is_epoch_end else "", res["n"], res["val_loss"], res["accuracy"],
+                 res["precision"], res["recall"], res["f1"], res["roc_auc"], res["pr_auc"], res["tn"], res["fp"], res["fn"],
+                 res["tp"], res["f1_optimal_threshold"], res["f1_at_optimal_threshold"], three, args.selection_metric, score,
+                 "NEW BEST -> best/" if improved else f"no improvement ({state.evals_without_improvement}/{args.patience})",
+                 res["eval_seconds"])
+        write_csv_row(epochs_csv, {"epoch": state.epoch, "step": state.step, "is_epoch_end": int(is_epoch_end),
+                                   "is_best": int(improved), **res}, EPOCHS_HEADER)
+
+    t_loop = t_inv_start = common.timer()
+    steps_this_invocation = 0
+    overhead_inv = 0.0   # eval + checkpoint seconds this invocation, excluded from the ETA's per-step cost
+    window_loss, window_rows, window_steps, t_window = 0.0, 0, 0, common.timer()
+    last_eval_step = state.step if saved else -1
+    exit_code = 0
+
+    def flush_time() -> None:
+        nonlocal t_loop
+        now = common.timer()
+        state.train_seconds += now - t_loop
+        t_loop = now
+
+    try:
+        while state.epoch < args.epochs and not state.stopped_by:
+            batches = epoch_batches(train_split.lengths, args.batch_size, args.seed, state.epoch)
+            loader = make_loader(train_split, batches[state.batch_in_epoch:], multiple, args.micro_batch)
+            for b, (idx, micro_batches) in enumerate(loader, start=state.batch_in_epoch):
+                n_rows = len(idx)
+                loss_acc = 0.0
+                for ids, mask, _y_bin, allowed in micro_batches:
+                    ids, mask, allowed = (t.to(device, non_blocking=True) for t in (ids, mask, allowed))
+                    n_sub = int(ids.shape[0])
+                    with autocast_ctx(device):
+                        logits = model(ids, mask)
+                    loss = compute_loss(logits, allowed, args.loss, weights, args.focal_gamma)
+                    (loss * (n_sub / n_rows)).backward()
+                    loss_acc += float(loss.detach()) * n_sub / n_rows
+                torch.nn.utils.clip_grad_norm_(model.parameters(), args.grad_clip)
+                opt.step()
+                opt.zero_grad(set_to_none=True)
+                sched.step()
+                state.step += 1
+                state.batch_in_epoch = b + 1
+                steps_this_invocation += 1
+                window_loss += loss_acc
+                window_rows += n_rows
+                window_steps += 1
+                if not math.isfinite(loss_acc):
+                    LOG.error("non-finite loss at step %d; stopping", state.step)
+                    state.stopped_by = "non_finite_loss"
+                    exit_code = 3
+
+                # Read the allocator only on CUDA; the CPU dev path reports zeros without touching torch.cuda.
+                vram = common.vram_stats() if device.type == "cuda" else CPU_VRAM
+                state.peak_reserved_gb = max(state.peak_reserved_gb, vram["max_reserved_gb"])
+                state.peak_allocated_gb = max(state.peak_allocated_gb, vram["max_allocated_gb"])
+                if device.type == "cuda" and vram["max_reserved_gb"] > float(vr["hard_cap_gb"]):
+                    LOG.error("VRAM hard cap breached: %.2f GiB reserved > %s GiB; lower --micro-batch. Saving last/ and aborting",
+                              vram["max_reserved_gb"], vr["hard_cap_gb"])
+                    state.stopped_by = "vram_cap"
+                    exit_code = 3
+                elif not band_warned and vram["max_reserved_gb"] > float(vr["target_gb_high"]):
+                    LOG.warning("reserved peak %.2f GiB is above the target band [%s, %s] GiB", vram["max_reserved_gb"],
+                                vr["target_gb_low"], vr["target_gb_high"])
+                    band_warned = True
+
+                if stop.requested:
+                    state.stopped_by = f"signal:{stop.requested}"
+                is_eval = args.eval_every > 0 and state.step % args.eval_every == 0
+                epoch_end = b == len(batches) - 1
+                hit_max = args.max_steps is not None and state.step >= args.max_steps
+                if hit_max and not state.stopped_by:
+                    state.stopped_by = "max_steps"
+                is_log = state.step % args.log_every == 0 or is_eval or epoch_end or bool(state.stopped_by)
+                row: dict[str, Any] | None = None
+                if is_log:
+                    now = common.timer()
+                    elapsed = state.train_seconds + (now - t_loop)
+                    per_step = (now - t_inv_start - overhead_inv) / max(1, steps_this_invocation)
+                    eta = per_step * max(0, total_steps - state.step)
+                    row = {"step": state.step, "epoch": state.epoch, "loss": round(window_loss / max(1, window_steps), 5),
+                           "lr": sched.get_last_lr()[0], "examples_per_s": round(window_rows / max(1e-9, now - t_window), 1),
+                           "max_alloc_gb": round(vram["max_allocated_gb"], 3), "max_reserved_gb": round(vram["max_reserved_gb"], 3),
+                           "elapsed_s": round(elapsed, 1), "eta_s": round(eta)}
+                    LOG.info("step %d/%d epoch %d loss %.4f lr %.2e %.1f ex/s alloc %.2f GiB reserved %.2f GiB elapsed %s ETA %s",
+                             state.step, total_steps, state.epoch, row["loss"], row["lr"], row["examples_per_s"],
+                             row["max_alloc_gb"], row["max_reserved_gb"], _hms(elapsed), _hms(eta))
+                    window_loss, window_rows, window_steps, t_window = 0.0, 0, 0, now
+
+                # Evaluate at the cadence, at every epoch end and when --max-steps ends the run; never
+                # after a signal (fast exit), a VRAM breach or a NaN model (it must not become best/).
+                do_eval = (is_eval or epoch_end or hit_max) and state.step != last_eval_step \
+                    and state.stopped_by in ("", "max_steps")
+                if do_eval:
+                    t_ev = common.timer()
+                    record_eval(epoch_end)
+                    overhead_inv += common.timer() - t_ev
+                    last_eval_step = state.step
+                    if row is not None:
+                        m3 = state.last_val.get("macro_f1_3way")
+                        row.update(val_loss=round(state.last_val["val_loss"], 5), val_f1=round(state.last_val["f1"], 5),
+                                   val_roc_auc=round(state.last_val["roc_auc"], 5),
+                                   val_macro_f1_3way=None if m3 is None or math.isnan(m3) else round(m3, 5))
+                    if state.evals_without_improvement >= args.patience and not state.stopped_by:
+                        state.stopped_by = "early_stopping"
+                        LOG.info("early stopping: %d evaluations without %s improvement (best %.4f at step %d)",
+                                 state.evals_without_improvement, args.selection_metric, state.best_score, state.best_step)
+                if row is not None:
+                    write_csv_row(metrics_csv, row, METRICS_HEADER)
+                    state.metrics_tail = (state.metrics_tail + [row])[-20:]
+                if state.stopped_by:
+                    break
+                if do_eval:
+                    flush_time()
+                    t_sv = common.timer()
+                    save_last(run_dir, model, opt, sched, state, args, schedule, device)
+                    overhead_inv += common.timer() - t_sv
+                    t_window = common.timer()   # examples/s must not absorb eval + checkpoint time
+            if not state.stopped_by:
+                state.batch_in_epoch = 0
+                state.epoch += 1
+                if state.epoch >= args.epochs:
+                    state.stopped_by = "epochs_done"
+    except KeyboardInterrupt:
+        LOG.error("second interrupt: hard stop without a checkpoint")
+        return 130
+
+    flush_time()
+    save_last(run_dir, model, opt, sched, state, args, schedule, device)
+    if args.cpu and torch.cuda.is_initialized():
+        # The --cpu contract is "the GPU is never touched"; a regression here would OOM the pretraining run.
+        LOG.error("--cpu run initialised CUDA; a code path touched torch.cuda on the CPU dev path (bug)")
+        exit_code = exit_code or 3
+
+    rec = experiment_record(args, state, params, steps_per_epoch)
+    exp_path = Path(cfg["paths"]["reports"]) / "experiments.jsonl"
+    how = upsert_experiment(exp_path, rec)
+    LOG.info("%s experiment record for %s in %s", how, args.run_name, exp_path)
+    if state.step >= RUN_LOG_MIN_STEPS:
+        common.append_run_log(run_log_block(args, state, run_dir, params))
+        LOG.info("appended a RUN_LOG.md entry")
+    v = state.best_val or {}
+    print(f"\nTotal training time: {state.train_seconds:.1f} s ({_hms(state.train_seconds)}) over {state.step} optimizer steps"
+          f" [{state.stopped_by}]; this invocation {common.timer() - t_invocation:.1f} s wall")
+    print(f"Peak VRAM reserved {state.peak_reserved_gb:.2f} GiB (allocated {state.peak_allocated_gb:.2f} GiB) on {device_info['device']}")
+    if v:
+        print(f"Best val by {args.selection_metric} (step {state.best_step}, epoch {state.best_epoch}), {UNWANTED_NAME} roll-up: "
+              + " ".join(f"{k}={v[k]:.4f}" for k in METRIC_KEYS) + f"  -> {run_dir / 'best'}")
+        if v.get("n_3way"):
+            print(f"Three-way (n={v['n_3way']}): macro-F1={v['macro_f1_3way']:.4f} accuracy={v['accuracy_3way']:.4f} "
+                  + " ".join(f"F1[{LABEL_NAMES[c]}]={v[f'f1_{s}']:.4f}" for c, s in enumerate(CLASS_SLUGS))
+                  + f"; phishing-vs-spam on true unwanted [pp ps sp ss]=[{v['pvs_phish_as_phish']} {v['pvs_phish_as_spam']} "
+                    f"{v['pvs_spam_as_phish']} {v['pvs_spam_as_spam']}]")
+    return exit_code
+
+
+def _hms(seconds: float) -> str:
+    s = int(round(seconds))
+    return f"{s // 3600:d}:{(s % 3600) // 60:02d}:{s % 60:02d}"
+
+
+# ------------------------------------------------------------------------------------ plot
+def plot(ns: argparse.Namespace, cfg: dict) -> int:
+    import matplotlib
+
+    matplotlib.use("Agg")
+    import matplotlib.pyplot as plt
+
+    run_dir = Path(cfg["paths"]["checkpoints"]) / "finetune" / ns.run_name
+    metrics_csv, epochs_csv = run_dir / "metrics.csv", run_dir / "epochs.csv"
+    if not metrics_csv.exists():
+        LOG.error("%s not found", metrics_csv)
+        return 2
+    m = _read_csv(metrics_csv)
+    e = _read_csv(epochs_csv) if epochs_csv.exists() else []
+    out = Path(ns.out) if ns.out else Path(cfg["paths"]["reports"]) / "finetune_curves.png"
+    out.parent.mkdir(parents=True, exist_ok=True)
+
+    # Two panels, one y-axis each (loss and scores live on different scales; no dual axis).
+    fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4.2), dpi=150)
+    steps = [float(r["step"]) for r in m]
+    ax1.plot(steps, [float(r["loss"]) for r in m], color=PALETTE[0], lw=2, label="train loss")
+    if e:
+        ax1.plot([float(r["step"]) for r in e], [float(r["val_loss"]) for r in e], color=PALETTE[1], lw=2,
+                 marker="o", ms=4, label="val loss (CE)")
+    ax1.set_xlabel("optimizer step")
+    ax1.set_ylabel("loss")
+    ax1.set_title(f"{ns.run_name}: loss")
+    ax1.legend(frameon=False)
+    if e:
+        es = [float(r["step"]) for r in e]
+        series = [("f1", "val F1 (unwanted roll-up)", PALETTE[0]), ("roc_auc", "val ROC-AUC", PALETTE[1]),
+                  ("pr_auc", "val PR-AUC", PALETTE[2])]
+        if any(_num(r.get("macro_f1_3way")) is not None for r in e):
+            series.append(("macro_f1_3way", "val macro-F1 (3-way)", PALETTE[3]))
+        lows = []
+        for key, label, color in series:
+            pts = [(s, _num(r.get(key))) for s, r in zip(es, e)]
+            pts = [(s, y) for s, y in pts if y is not None]
+            if not pts:
+                continue
+            ax2.plot([p[0] for p in pts], [p[1] for p in pts], color=color, lw=2, marker="o", ms=4, label=label)
+            lows.append(min(p[1] for p in pts))
+        best = max(e, key=lambda r: float(r["f1"]))
+        ax2.annotate(f"best F1 {float(best['f1']):.4f}\nstep {int(float(best['step']))}", (float(best["step"]), float(best["f1"])),
+                     textcoords="offset points", xytext=(8, -28), fontsize=8, color="#444444")
+        ax2.set_ylim(max(0.0, min(0.5, min(lows) - 0.05)) if lows else 0.0, 1.0)
+        ax2.legend(frameon=False, loc="lower right")
+    else:
+        ax2.text(0.5, 0.5, "no evaluations yet", ha="center", va="center", transform=ax2.transAxes)
+    ax2.set_xlabel("optimizer step")
+    ax2.set_ylabel("score")
+    ax2.set_title(f"{ns.run_name}: validation")
+    for ax in (ax1, ax2):
+        ax.grid(True, color="#e6e6e6", lw=0.6)
+        for side in ("top", "right"):
+            ax.spines[side].set_visible(False)
+    fig.tight_layout()
+    fig.savefig(out)
+    LOG.info("wrote %s (%d train rows, %d eval rows)", out, len(m), len(e))
+    print(out)
+    return 0
+
+
+def _read_csv(path: Path) -> list[dict[str, str]]:
+    with path.open(newline="", encoding="utf-8") as fh:
+        return [r for r in csv.DictReader(fh) if r.get("step")]
+
+
+def _num(s: str | None) -> float | None:
+    """CSV cell -> float, None for blank / NaN (a binary run has no three-way columns)."""
+    if s is None or s == "":
+        return None
+    try:
+        v = float(s)
+    except ValueError:
+        return None
+    return None if math.isnan(v) else v
+
+
+# ------------------------------------------------------------------------------------ list
+def list_experiments(ns: argparse.Namespace, cfg: dict) -> int:
+    path = Path(cfg["paths"]["reports"]) / "experiments.jsonl"
+    if not path.exists():
+        print(f"no experiments yet ({path})")
+        return 0
+    runs: dict[str, dict[str, Any]] = {}
+    for line in path.read_text(encoding="utf-8").splitlines():
+        if not line.strip():
+            continue
+        rec = json.loads(line)
+        slot = runs.setdefault(rec["run_name"], {})
+        slot["test" if rec.get("test") is not None else "train"] = rec
+
+    def f(x: Any, nd: int = 4) -> str:
+        return "-" if x is None or (isinstance(x, float) and math.isnan(x)) else (f"{x:.{nd}f}" if isinstance(x, float) else str(x))
+
+    # "val acc/P/R/F1" are the Unwanted (phishing or spam) roll-up at 0.5, identical to the binary runs' numbers;
+    # the three-way columns are blank for binary (num_labels 2) runs.
+    head = ["run", "init", "classes", "layers", "frozen", "epochs run/planned", "loss", "lr", "batch", "synthetic", "params (M)",
+            "train min", "peak GiB", "best ep", "val acc", "val P", "val R", "val F1", "val ROC-AUC", "val PR-AUC",
+            "val macro-F1 3w", "val P phish", "val R phish", "test F1", "test ROC-AUC", "stopped by"]
+    rows = []
+    for name, slot in runs.items():
+        tr, te = slot.get("train"), slot.get("test")
+        base = tr or te
+        d = base.get("dials", {})
+        v = (tr or {}).get("val") or {}
+        t = (te or {}).get("test") or {}
+        phish = (v.get("per_class") or {}).get(LABEL_NAMES[1]) or {}
+        init = base.get("init", "-")
+        init = "scratch" if init == "scratch" else "pretrained" if init.startswith("pretrained") else init
+        rows.append([name, init, f(d.get("num_labels", 2)), f(d.get("n_layers")), f(d.get("frozen_layers")),
+                     f"{f(d.get('epochs_run'), 2)}/{f(d.get('epochs_planned'))}", f(d.get("loss")),
+                     f"{d['lr']:.0e}" if d.get("lr") else "-", f(d.get("batch_size")),
+                     f"{d.get('synthetic', '-')}" + (f" ({d['synthetic_rows']})" if d.get("synthetic_rows") else ""),
+                     f((base.get("params") or {}).get("total", 0) / 1e6, 1) if base.get("params") else "-",
+                     f((tr or {}).get("train_seconds", 0) / 60.0, 1) if tr else "-", f((tr or {}).get("peak_vram_gb"), 2),
+                     f((tr or {}).get("best_epoch")), f(v.get("accuracy")), f(v.get("precision")), f(v.get("recall")),
+                     f(v.get("f1")), f(v.get("roc_auc")), f(v.get("pr_auc")), f(v.get("macro_f1_3way")),
+                     f(phish.get("precision")), f(phish.get("recall")), f(t.get("f1")), f(t.get("roc_auc")),
+                     f((tr or {}).get("stopped_by"))])
+    print("| " + " | ".join(head) + " |")
+    print("|" + "|".join("---" for _ in head) + "|")
+    for r in rows:
+        print("| " + " | ".join(r) + " |")
+    print(f"\n{len(rows)} run(s) from {path}")
+    return 0
+
+
+# ------------------------------------------------------------------------------------- CLI
+def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--config", default="config.yaml")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+
+    tr = sub.add_parser("train", help="fine-tune the classifier (dials default to config.yaml; --resume restores the saved ones)")
+    tr.add_argument("--run-name", default="finetune")
+    tr.add_argument("--init", default=None, help="pretrained | pretrained:<dir> | scratch (default: newest checkpoints/pretrain/*/final)")
+    tr.add_argument("--n-layers", type=int, default=None, help="use only the first N encoder layers")
+    tr.add_argument("--freeze-layers", type=int, default=None, help="freeze embeddings + first K layers")
+    tr.add_argument("--loss", choices=LOSSES, default=None)
+    tr.add_argument("--focal-gamma", type=float, default=None)
+    tr.add_argument("--lr", type=float, default=None)
+    tr.add_argument("--batch-size", type=int, default=None, help="logical batch (rows per optimizer step)")
+    tr.add_argument("--micro-batch", type=int, default=None, help="rows per fwd/bwd; batch-size must be a multiple (default 16)")
+    tr.add_argument("--epochs", type=int, default=None)
+    tr.add_argument("--max-steps", type=int, default=None, help="dev: stop after this many optimizer steps")
+    tr.add_argument("--eval-every", type=int, default=None, help="optimizer steps between full val evaluations")
+    tr.add_argument("--log-every", type=int, default=None)
+    tr.add_argument("--eval-batch-size", type=int, default=None)
+    tr.add_argument("--synthetic", choices=("on", "off"), default=None, help="append data/processed/synthetic.parquet to TRAIN")
+    tr.add_argument("--limit-rows", type=int, default=None, help="dev: seeded subsample of train AND val")
+    tr.add_argument("--seed", type=int, default=None)
+    tr.add_argument("--num-labels", type=int, choices=NUM_LABELS_ALLOWED, default=None,
+                    help="override finetune.num_labels: 3 = legitimate/phishing/spam (partial-label loss), 2 = binary baseline")
+    tr.add_argument("--cpu", action="store_true", help="dev only: run on the CPU with the config as given (tiny encoder + fixtures)")
+    tr.add_argument("--resume", action="store_true",
+                    help="continue <run>/last (dials restored; CLI flags override, except --init/--n-layers/--freeze-layers/"
+                         "--batch-size/--seed/--num-labels which are refused; --lr is re-applied to the restored optimizer)")
+    tr.add_argument("--overwrite", action="store_true", help="discard an existing run dir of the same name")
+
+    pl = sub.add_parser("plot", help="train loss and val F1/AUC vs step -> reports/finetune_curves.png")
+    pl.add_argument("--run-name", required=True)
+    pl.add_argument("--out", default=None)
+
+    sub.add_parser("list", help="reports/experiments.jsonl as a markdown table")
+    return ap.parse_args(argv)
+
+
+def main(argv: list[str] | None = None) -> int:
+    ns = parse_args(argv)
+    cfg = common.load_config(ns.config)
+    global LOG
+    LOG = common.setup_logging("finetune", cfg["paths"]["logs"])
+    if ns.cmd == "train":
+        return train(ns, cfg)
+    if ns.cmd == "plot":
+        return plot(ns, cfg)
+    return list_experiments(ns, cfg)
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/presentation/src/make_synthetic.py b/presentation/src/make_synthetic.py
new file mode 100644
index 0000000000000000000000000000000000000000..49eb4e959b1363c163e5ec9ee51922cf3624b6bc
--- /dev/null
+++ b/presentation/src/make_synthetic.py
@@ -0,0 +1,938 @@
+#!/usr/bin/env python
+"""Template-generated synthetic supplement for phishing fine-tuning (TRAINING DATA ONLY).
+
+What it is
+  A small (<= config data.synthetic.max_rows = 2,000), label-balanced set of short emails in the
+  register a small-business IT consultancy's clients actually see. Positive rows (label 1) imitate the
+  lure categories those clients get targeted with; negative rows (label 0) are ordinary business mail
+  written in the same tone so the model cannot shortcut on register alone. Every row is fabricated
+  from schematic templates plus seeded random slot filling -- no network, no LLM calls, no real
+  messages. Organisations and people are placeholders ("Northwind Dental", a fixed fictional name
+  list); every domain sits under the reserved .example / .invalid TLDs, IPs use the RFC 5737
+  documentation ranges and phone numbers use 555-01xx. Rows carry source="synthetic", doc_id
+  "synthetic:NNNNNN", cluster_id -1, and the parquet schema carries
+  {"generator": "make_synthetic.py", "note": "template-generated training supplement"}. The file is
+  never a deliverable and must never be presented as real mail.
+
+How it is used and reported
+  * finetune.py --synthetic on concatenates data/processed/synthetic.parquet onto finetune_train ONLY.
+    It never enters finetune_val (model selection / early stopping stay on real data), the TREC-06
+    holdout, the tokenizer corpus or MLM pretraining.
+  * Every headline metric is reported twice -- one run with and one without the supplement, identical
+    dials otherwise -- and reports/experiments.jsonl records dials.synthetic so the comparison is
+    auditable. The supplement is ~1.3% of the 157k training rows; its job is to nudge the model toward
+    the business-email-compromise / Microsoft 365 lure register that the 2005-2008 spam corpora lack,
+    not to move aggregate numbers. Any change on the holdout is reported as-is, including none.
+
+Shortcut control
+  Templated data invites the classifier to learn the template rather than the phenomenon. The
+  generator therefore
+  (a) draws greetings, sign-offs, signatures, disclaimers, HTML wrappers, subject prefixes, casing,
+      TLDs, URL schemes and the per-row length target from pools shared by both labels;
+  (b) puts a link in BOTH classes at comparable rates: every negative category carries a {url} slot
+      pointing at a plausible internal / vendor portal whose host prefixes (login., secure., docs.,
+      payments., *.sharepoint.*) and paths (/login, /auth/, /docs/, /inv/, ?id=) overlap the lookalike
+      vocabulary used by the lures, so neither "contains a URL", "<a href" nor the URL sub-tokens
+      (https, http, login, secure, doc, id, ...) separate the labels;
+  (c) writes benign sentences on label 0 that reuse lure vocabulary (password, verify, urgent, overdue,
+      DocuSign, OneDrive, wire, gift cards, expires, suspended, immediately, authenticator, approve)
+      and mundane sentences on label 1 that reuse notice vocabulary (planned maintenance, posting for
+      awareness, we're, just), so the words themselves are not label-predictive;
+  (d) measures the result over the WHOLE vocabulary: `build` logs per-token label purity (document
+      frequency over lower-cased word tokens) for the 30 most frequent tokens, and flags EVERY token
+      with support >= 50 that is > 95% one class. The flagged list is written to
+      <stem>_report.json ("token_purity_flagged") and logged, so a residual shortcut is auditable
+      rather than hidden by a top-N cut-off. Exact duplicates are removed on common.normalize_text,
+      so the written file's duplicate rate is 0.
+
+Three-way labels (label3 / label_set)
+  The classifier now outputs Legitimate / Phishing / Spam. Every row carries, beside the binary
+  `label`, a `label3` int64 (0 legitimate, 1 phishing; never 2 spam or -1 unwanted here) and a
+  `label_set` string ("legit" / "phish"). The mapping is by category, i.e. by the "<label>:<category>"
+  keys of the report: every "1:*" category -> label3 1 / "phish", every "0:*" category -> label3 0 /
+  "legit". This is correct by construction, not by assumption: all eight positive templates
+  (m365_signin, mfa_prompt, helpdesk_quota, fake_fileshare, invoice_lure, bec_payment, bec_giftcard,
+  vendor_bankchange) are credential-harvesting, account-takeover or business-email-compromise lures --
+  targeted deception aimed at a credential, a payment or a gift-card code -- and none is bulk
+  unsolicited advertising, which is what "spam" means in the label contract. The synthetic set therefore
+  contributes NO spam rows; spam supervision comes from the real SpamAssassin / Enron-Spam folders.
+  PHISH_CATEGORIES pins that list: a new positive template outside it makes `build` fail rather than be
+  silently filed as phishing. Adding the two columns does not touch the RNG stream (they are derived
+  after generation), so with the same seed every existing column -- doc_id, text, subject, body,
+  label, category -- is byte-identical to the binary-era file and earlier experiments stay reproducible.
+
+Length
+  Each row draws a target length from a right-skewed triangular distribution over 50-200 words
+  (mode at 50) and pads toward it from the shared filler pool, so the supplement spans a band of
+  lengths instead of a 50-70 word spike. Bodies are hard-capped at 400 words.
+
+Subcommands
+  build [--n N] [--out PATH] [--seed S]   write the parquet + <stem>_report.json beside it
+  show  [--k 6] [--path PATH]             print k sample rows per class
+"""
+
+from __future__ import annotations
+
+import argparse
+import logging
+import random
+import re
+import sys
+from collections import Counter
+from dataclasses import dataclass, field
+from datetime import date, timedelta
+from pathlib import Path
+from typing import Any
+
+import pandas as pd
+import pyarrow as pa
+import pyarrow.parquet as pq
+
+sys.path.insert(0, str(Path(__file__).parent))
+import common  # noqa: E402
+
+LOG = logging.getLogger("make_synthetic")
+
+COLUMNS = ["doc_id", "text", "subject", "body", "label", "source", "cluster_id", "category", "label3", "label_set"]
+
+# Three-way label contract (shared with data_prep / finetune / evaluate): 0 legitimate, 1 phishing, 2 spam.
+# The synthetic generator only emits 0 and 1 -- see "Three-way labels" in the module docstring.
+LABEL3_NAMES = {0: "Legitimate", 1: "Phishing", 2: "Spam"}
+LABEL_SET_OF = {0: "legit", 1: "phish", 2: "spam"}
+# Every positive template is a credential / account-takeover / BEC lure, hence phishing, not spam.
+PHISH_CATEGORIES = frozenset({"m365_signin", "mfa_prompt", "helpdesk_quota", "fake_fileshare", "invoice_lure",
+                              "bec_payment", "bec_giftcard", "vendor_bankchange"})
+PARQUET_META = {"generator": "make_synthetic.py", "note": "template-generated training supplement"}
+MIN_WORDS, TARGET_MAX_WORDS, MAX_WORDS = 50, 200, 400
+PURITY_SUPPORT, PURITY_MAX, PURITY_TOP_N = 50, 0.95, 30
+TOKEN_RE = re.compile(r"[a-z][a-z0-9']+")
+
+# ----------------------------------------------------------------------------- placeholder world
+ORGS = [
+    "Northwind Dental", "Cedar Ridge Plumbing", "Harborview CPA", "Maple Street Veterinary",
+    "Bluewater Landscaping", "Summit Peak Physical Therapy", "Oak & Ivy Law Group", "Riverbend Bakery",
+    "Pinecrest Family Chiropractic", "Lakeshore Realty Partners", "Ironwood Machine Works",
+    "Willow Creek Pediatrics", "Copper Kettle Catering", "Foxglove Interior Design",
+]
+VENDORS = [
+    "Tri-County Office Supply", "Copperleaf Dental Lab", "Evergreen Payroll Services", "Keystone Janitorial",
+    "Silverline Medical Supply", "Redwood Print & Sign", "Granite Coast Insurance", "Brookside Uniform Co.",
+    "Clearpath Telecom", "Hollow Oak Accounting",
+]
+CLIENTS = [
+    "Juniper Hills HOA", "Starling Logistics", "Meadowbrook Assisted Living", "Bramble & Finch Architects",
+    "Sandstone Auto Repair", "Quarry Lane Brewing",
+]
+MSPS = ["Brightline IT Partners", "Signal Harbor Technology", "Northgate Managed IT"]
+FIRST = [
+    "Avery", "Jordan", "Morgan", "Casey", "Riley", "Taylor", "Jamie", "Quinn", "Reese", "Dana", "Elliot",
+    "Harper", "Rowan", "Sasha", "Devon", "Marisol", "Tobias", "Priya", "Lena", "Malik", "Ingrid", "Theo",
+    "Noor", "Callum", "Bea", "Rafael", "Wren", "Odette", "Kofi", "Imani",
+]
+LAST = [
+    "Calloway", "Brightwater", "Fenwick", "Ashdown", "Merriweather", "Okafor", "Lindqvist", "Talbot",
+    "Pemberton", "Quill", "Hartsell", "Dunmore", "Ravensworth", "Kilbride", "Marchetti", "Holloway", "Nakamura",
+]
+OWNER_TITLES = ["Owner", "Practice Owner", "Managing Partner", "President", "CEO", "Founder", "Principal"]
+STAFF_TITLES = [
+    "Office Manager", "Practice Administrator", "Bookkeeper", "Operations Lead", "Front Desk Coordinator",
+    "Accounts Payable", "Project Coordinator", "Senior Associate",
+]
+PROJECTS = [
+    "website refresh", "server migration", "new patient portal", "office network upgrade", "kitchen remodel bid",
+    "Q4 tax planning engagement", "warehouse Wi-Fi survey", "phone system cutover", "brand refresh",
+    "file share cleanup",
+]
+FILES = [
+    "Q3_Payroll_Summary.xlsx", "Signed_Agreement_2026.pdf", "Remittance_Advice.pdf", "Employee_Handbook_v4.docx",
+    "Statement_of_Work.pdf", "Bonus_Schedule.xlsx", "Vendor_W9.pdf", "Project_Timeline.xlsx",
+    "Insurance_Renewal.pdf", "Board_Minutes.docx", "Quote_Revised.pdf", "Purchase_Order.pdf",
+]
+LOCATIONS = ["Frankfurt, Germany", "Toronto, Canada", "Sao Paulo, Brazil", "Singapore", "an unrecognized location",
+             "Warsaw, Poland", "Dallas, TX", "Amsterdam, Netherlands"]
+DEVICES = ["iPhone", "Windows laptop", "Android phone", "MacBook", "front desk PC", "iPad"]
+TLDS = ["example", "example", "invalid"]
+SCHEMES = ["https://", "https://", "http://"]
+LOOKALIKES = [
+    "micr0soft-365-login", "login-microsoftonline-secure", "office365-mailbox-center", "entra-id-auth",
+    "sharepoint-docs-access", "onedrive-filetransfer", "docusign-envelope-review", "m365-quota-admin",
+    "secure-invoice-portal", "outlook-web-sync", "doc-share-cloud", "payments-remit-portal",
+]
+# Legitimate hosts reuse the lookalikes' vocabulary (login, secure, docs, payments, sharepoint, portal)
+# under the org's / vendor's / MSP's own placeholder domain, the way real tenants do.
+LEGIT_HOST_PREFIXES = ["portal", "intranet", "billing", "status", "support", "login", "secure", "docs",
+                       "payments", "mail", "tickets", "hr", "calendar"]
+LEGIT_PATHS = ["/billing", "/hr/benefits", "/tickets/", "/calendar", "/projects/", "/docs/", "/", "/status",
+               "/login", "/auth/", "/inv/", "/s/", "/review?id=", "/sites/", "/personal/", "/tickets?id=",
+               "/invoices/", "/quote/", "/m/", "/doc/"]
+PHISH_PATHS = ["/auth/", "/s/", "/review?id=", "/login/", "/doc/", "/session/", "/m/", "/inv/", "/docs/",
+               "/billing", "/sites/", "/tickets?id="]
+GENERIC_ANCHORS = ["click here", "here", "Open in browser", "View details", "Open"]
+
+
+def slug(name: str) -> str:
+    return re.sub(r"-+", "-", re.sub(r"[^a-z0-9]+", "-", name.lower().replace("&", "and"))).strip("-")
+
+
+def money(rng: random.Random, lo: float, hi: float) -> str:
+    return f"${rng.uniform(lo, hi):,.2f}"
+
+
+def fmt_date(d: date, rng: random.Random) -> str:
+    style = rng.randrange(4)
+    if style == 0:
+        return d.strftime("%A, %B ") + str(d.day)
+    if style == 1:
+        return d.strftime("%m/%d/%Y")
+    if style == 2:
+        return d.strftime("%b ") + str(d.day)
+    return d.strftime("%B ") + str(d.day) + d.strftime(", %Y")
+
+
+def _token(rng: random.Random) -> str:
+    return "".join(rng.choice("abcdefghjkmnpqrstuvwxyz23456789") for _ in range(rng.randrange(6, 14)))
+
+
+def make_ctx(rng: random.Random) -> dict[str, str]:
+    """One fictional scenario: who sends what to whom. Every value is a placeholder.
+
+    Both URL flavours are built from the same scheme pool and the same path/prefix vocabulary so that
+    URL presence and URL sub-tokens carry no label signal; only the host's relationship to the sender
+    (own domain vs. lookalike) differs, which is the cue a classifier should actually learn.
+    """
+    org = rng.choice(ORGS)
+    org_dom = f"{slug(org)}.{rng.choice(TLDS)}"
+    vendor = rng.choice(VENDORS)
+    vendor_dom = f"{slug(vendor)}.{rng.choice(TLDS)}"
+    msp = rng.choice(MSPS)
+    msp_dom = f"{slug(msp)}.{rng.choice(TLDS)}"
+    names = rng.sample(FIRST, 6)
+    lasts = rng.sample(LAST, 6)
+    base = date(2026, 1, 5) + timedelta(days=rng.randrange(0, 360))
+    d2 = base + timedelta(days=rng.randrange(1, 15))
+    hour = rng.choice([8, 9, 10, 11, 13, 14, 15, 16, 21, 22])
+    t = f"{hour % 12 or 12}:{rng.choice(['00', '15', '30', '45'])} {'AM' if hour < 12 else 'PM'}"
+    if rng.random() < 0.25:
+        phish_host = f"{slug(org)}-{rng.choice(['it-support', 'helpdesk', 'payroll', 'secure'])}.{rng.choice(TLDS)}"
+    else:
+        phish_host = f"{rng.choice(LOOKALIKES)}.{rng.choice(TLDS)}"
+    phish_path = rng.choice(PHISH_PATHS)
+    own_dom = rng.choice([org_dom, org_dom, vendor_dom, msp_dom])
+    r = rng.random()
+    if r < 0.15:
+        legit_host = f"{slug(org).split('-')[0]}.sharepoint.{rng.choice(TLDS)}"
+    elif r < 0.25:
+        legit_host = f"{slug(org).split('-')[0]}-my.sharepoint.{rng.choice(TLDS)}"
+    elif r < 0.35:
+        legit_host = own_dom
+    else:
+        legit_host = f"{rng.choice(LEGIT_HOST_PREFIXES)}.{own_dom}"
+    legit_path = rng.choice(LEGIT_PATHS)
+    used = rng.randrange(46, 50) + rng.random()
+    return {
+        "org": org, "org_dom": org_dom, "org_short": org.split()[0],
+        "first": names[0], "last": lasts[0], "email": f"{names[0].lower()}@{org_dom}",
+        "ceo_first": names[1], "ceo_last": lasts[1], "ceo_title": rng.choice(OWNER_TITLES),
+        "col_first": names[2], "col_last": lasts[2], "col_title": rng.choice(STAFF_TITLES),
+        "hr_first": names[3], "hr_last": lasts[3],
+        "vendor": vendor, "vendor_dom": vendor_dom, "ven_first": names[4], "ven_last": lasts[4],
+        "msp": msp, "msp_dom": msp_dom, "tech_first": names[5], "tech_last": lasts[5],
+        "client": rng.choice(CLIENTS), "project": rng.choice(PROJECTS), "file": rng.choice(FILES),
+        "amount": money(rng, 380, 48_000), "amount2": money(rng, 90, 2_500),
+        "inv": f"{rng.choice(['INV-', 'INV', '#', 'No. ', 'A'])}{rng.randrange(10_000, 99_999)}",
+        "po": f"PO-{rng.randrange(1000, 9999)}", "acct4": f"{rng.randrange(0, 9999):04d}",
+        "date": fmt_date(base, rng), "date2": fmt_date(d2, rng), "time": t,
+        "hours": str(rng.choice([2, 4, 12, 24, 48])), "days": str(rng.choice([3, 5, 7, 10, 14, 30])),
+        "ticket": f"#{rng.randrange(10_000, 99_999)}", "location": rng.choice(LOCATIONS),
+        "device": rng.choice(DEVICES),
+        "ip": f"{rng.choice(['203.0.113', '198.51.100', '192.0.2'])}.{rng.randrange(2, 254)}",
+        "code": f"{rng.randrange(0, 999_999):06d}", "gb_used": f"{used:.1f}",
+        "gb_total": str(rng.choice([50, 50, 100])), "pct": str(rng.choice([92, 95, 97, 98, 99])),
+        "phone": f"(555) 01{rng.randrange(0, 100):02d}", "ext": str(rng.randrange(100, 400)),
+        "gift_n": str(rng.choice([4, 5, 6, 8, 10])), "gift_v": rng.choice(["$100", "$200", "$250", "$500"]),
+        "room": rng.choice(["the conference room", "Room B", "Teams", "Zoom", "the upstairs office"]),
+        "pages": str(rng.randrange(2, 14)),
+        "phish_url": f"{rng.choice(SCHEMES)}{phish_host}{phish_path}{_token(rng)}",
+        "legit_url": f"{rng.choice(SCHEMES)}{legit_host}{legit_path}"
+                     + (_token(rng) if legit_path.endswith(("/", "=")) else ""),
+    }
+
+
+# ----------------------------------------------------------------------------- shared pools (both labels)
+GREETINGS = ["Hi {first},", "Hello {first},", "{first},", "Good morning {first},", "Dear {first},", "Hi there,",
+             "Hello,", "Dear {first} {last},", "Hey {first} -", "Team,", "Hi all,", "Good afternoon,", ""]
+SIGNOFFS = ["Thanks,", "Thank you,", "Best,", "Regards,", "Kind regards,", "Thanks!", "Cheers,",
+            "Sent from my iPhone", "Best regards,", "Many thanks,", "Talk soon,", ""]
+# Padding drawn by BOTH labels. Deliberately mixes mundane office lines with security-awareness lines
+# that carry lure vocabulary (password, verify, wire, gift cards, urgent, expires, suspended, approve) in
+# a benign frame, and notice vocabulary (planned maintenance, posting for awareness) in a neutral frame.
+SHARED_FILLER = [
+    "Also, a reminder that the office closes early on Friday for the staff lunch.",
+    "Please let me know if you have any questions.",
+    "If this has already been handled, please disregard this message.",
+    "I am out part of tomorrow but will be checking email.",
+    "Feel free to forward this to anyone on the team who needs it.",
+    "We appreciate your business and your patience.",
+    "Let me know if another time works better for you.",
+    "Apologies for the short notice on this one.",
+    "Reply to this email or call {phone} ext. {ext} if you need anything.",
+    "Thanks again for getting back to me so quickly last week.",
+    "As always, reach out with any concerns.",
+    "Hope your week is going well.",
+    "I will follow up again on {date2} if I do not hear back.",
+    "Copying {col_first} so they are in the loop.",
+    "Reminder: IT will never ask for your password, verification codes or gift card numbers by email.",
+    "If anything in a message looks off, verify it by calling the sender at a number you already have.",
+    "Our bank and remittance details only change in writing on letterhead, confirmed by phone.",
+    "Wire and ACH requests over {amount2} need a second approver, however urgent they seem.",
+    "Urgent requests still go through the normal ticket queue; nothing here needs you to reply immediately.",
+    "Your login and authenticator settings are unchanged unless you hear from {tech_first} directly.",
+    "Anything that expires or gets suspended is announced at least {days} days ahead.",
+    "The shared OneDrive and DocuSign folders are the only places we keep signed documents.",
+    "Invoices are paid on {days}-day terms; a genuine overdue notice always comes with a phone call.",
+    "Accounts Payable approves payments on {date2}; please have receipts in by then.",
+    "We're keeping the planned maintenance window short, so expect only a brief Teams interruption.",
+    "Posting this for awareness; the full notes are on the intranet.",
+    "Just a heads up that the parking lot is being resurfaced on {date2}.",
+    "The new coffee machine arrives {date2}, so the break room will be closed that morning.",
+    "Quarterly numbers go to {ceo_first} by {date2}; send me anything you want included.",
+    "The {project} kickoff moved to {date2}; same room, same time.",
+    "Please keep your mailbox under quota; archive anything older than a year.",
+    "Held messages release automatically once the mail filter finishes its scan.",
+    "{col_first} is covering the front desk today, so route walk-ins to them.",
+    "Please confirm you received this so I can check it off the list.",
+    "The team calendar and the status page are the quickest ways to track what's going on.",
+    "The PTO form and the benefits enrollment form are both on the intranet under Human Resources.",
+    "Here's a copy of the agenda and the recap from the last meeting for reference.",
+    "Appointment and booking changes go through the front desk; reschedule there, not by email.",
+    "The estimate and pricing history for the {project} are in the shared folder - it's the usual place.",
+    "Our banking and remittance details have not changed; remit to the address on the invoice.",
+    "A few items are still open from last month's advisory; the resolved ones are posted already.",
+    "Within a few days you should get the final document and the instructions to continue.",
+    "Session storage and incoming mail activity were normal; the service health page says resolved.",
+    "The IT team will be in {room} on {date} if you'd prefer to do this in person.",
+    "This step replaces the paper form previously handled by {hr_first} in HR.",
+    "Thanks for keeping your details current; nothing about your identity or codes needs to change.",
+    "If a link in any email looks off, don't click it - call the sender and avoid guessing.",
+    "Keep it quiet until it's done; the client would rather get the news from us directly.",
+    "You can get more information about the proposal from {col_first} when they are available.",
+    "The admin center will show progress on the migration; dates may move a little along the way.",
+    "Direct deposit dates are posted; the deposit for the {project} is covered in the quote.",
+    "Coverage for the front desk starts at {time}; {col_first} confirms the swap via email.",
+    "Mailboxes over {gb_total} GB bounce new mail, so please validate your archive settings and process old items.",
+    "Credentials for the new system go out from {tech_first} on {date2}; nothing is affected before then.",
+    "Elections for the benefits plan are processed by {hr_first}; move anything else to the HR queue.",
+    "Deactivation of old accounts happens {days} days after someone leaves, never the same day.",
+    "The nightly backup will start at {time}; expect a couple of calls about slow file access until it finishes.",
+    "We do not change banks without a phone call first, so if you ever see a monthly statement from a new account, ask.",
+]
+DISCLAIMERS = [
+    "CONFIDENTIALITY NOTICE: This message and any attachments are intended only for the named recipient.",
+    "This email was sent to {email} because of your account with {org}.",
+    "Please consider the environment before printing this email.",
+    "This mailbox is not monitored; please do not reply directly to this message.",
+    "If you received this message in error, please notify the sender and delete it.",
+]
+SUBJECT_PREFIX = ["", "", "", "", "RE: ", "Re: ", "FW: ", "Fwd: ", "[External] ", "ACTION: ", "Reminder: "]
+
+
+@dataclass
+class Category:
+    """A template family. `parts` is a list of slots; each slot lists interchangeable sentences (a slot
+    may contain "" to be optional). `extras` are optional padding paragraphs; `signers` name signature
+    styles. `url` selects which link pool (if any) is woven in; `anchors` is the category-appropriate
+    link text used in HTML bodies (plain-text bodies carry the bare URL because the template sentence
+    already supplies the prefix)."""
+
+    label: int
+    subjects: list[str]
+    parts: list[list[str]]
+    extras: list[str]
+    signers: list[str]
+    url: str = "phish"            # "phish" | "legit" | "none"
+    html_prob: float = 0.25
+    anchors: list[str] = field(default_factory=lambda: list(GENERIC_ANCHORS))
+
+
+SIGNATURES = {
+    "ceo": ["{ceo_first}", "{ceo_first} {ceo_last}\n{ceo_title}, {org}", "{ceo_first} {ceo_last}\n{ceo_title}",
+            "{ceo_first}\nSent from my iPhone", "- {ceo_first}"],
+    "colleague": ["{col_first}", "{col_first} {col_last}\n{col_title}\n{org}",
+                  "{col_first} {col_last} | {col_title} | {phone}"],
+    "hr": ["{hr_first} {hr_last}\nHuman Resources, {org}", "{hr_first}\nHR & Payroll", "{org} Human Resources"],
+    "vendor": ["{ven_first} {ven_last}\nAccounts Receivable\n{vendor}\n{phone}", "{vendor} Billing Department",
+               "{ven_first} {ven_last} | {vendor}", "{ven_first}\n{vendor}"],
+    "msp": ["{tech_first} {tech_last}\nService Desk, {msp}\n{phone}", "{msp} Support Team",
+            "{tech_first}\n{msp} | support@{msp_dom}", "The {msp} Service Desk"],
+    "brand": ["Microsoft 365 Message center", "Microsoft account team", "The Microsoft 365 Team",
+              "Microsoft Security", "IT Service Desk", "Microsoft Entra ID"],
+    "itdesk": ["IT Help Desk\n{org}", "{org} IT Department", "Mail Administrator", "System Administrator",
+               "IT Support Team | {phone}", "The {msp} Service Desk", "Helpdesk"],
+    "docs": ["OneDrive", "Shared via OneDrive for Business", "DocuSign Electronic Signature Service",
+             "SharePoint Online", "{col_first} {col_last} (via OneDrive)", "Document Services"],
+    "client": ["{ven_first} {ven_last}\n{client}", "{ven_first}\nOperations, {client}", "{ven_first} | {client}"],
+}
+
+# Optional trailing slot for every positive category: mundane notice vocabulary (planned maintenance,
+# posting for awareness, we're, just, by) so those words are not negative-only.
+PHISH_MUNDANE = [
+    "We're posting this for awareness ahead of the planned maintenance on {date2}.",
+    "Just a reminder that this is part of the scheduled {org} security maintenance.",
+    "Add the maintenance window to your calendar; the status page will show progress.",
+    "Benefits and payroll questions still go to {hr_first} in Human Resources as usual.",
+    "Check the status of this request at the help desk if you have any questions.",
+    "The IT team will be in {room} on {date} if you'd prefer to do this in person.",
+    "This step replaces the paper form previously handled by {hr_first} in HR.",
+    "", "", "", "",
+]
+
+C: dict[str, Category] = {}
+
+# ------------------------------------------------------------------ positives (label 1)
+# Lure sentences are deliberately short and schematic; the realism lives in the varied placeholders,
+# not in persuasive copy. Each positive category pairs with a negative category in the same register.
+C["m365_signin"] = Category(1, [
+    "Your password expires today", "Action required: Microsoft 365 password expiration",
+    "Password expiry notice for {email}", "Your Microsoft 365 account will be suspended",
+    "Unusual sign-in activity on your account", "Microsoft 365: verification needed",
+    "Account review required within {hours} hours", "Sign-in blocked for {email}",
+], [
+    ["Your Microsoft 365 password for {email} expires today.",
+     "Your Office 365 credentials have not been updated under the new {org} security policy.",
+     "A sign-in from {location} (IP {ip}) was blocked on {date} at {time}.",
+     "Microsoft Entra ID flagged your account for a mandatory review."],
+    ["Sign in to keep your current password and avoid interruption: {url}",
+     "Confirm your account at the secure page to keep mailbox access: {url}",
+     "Re-authenticate now: {url}", "Review the activity and secure your account here: {url}"],
+    ["This link expires in {hours} hours.", "Failure to verify will suspend access to Outlook and Teams.",
+     "If you do not recognize this activity, act within {hours} hours.",
+     "Your usual password will stop working after that; check your spam folder if the code did not arrive.", ""],
+    PHISH_MUNDANE,
+], [], ["brand", "itdesk"], anchors=["Sign in", "Keep my password", "Review activity", "Verify account"])
+
+C["mfa_prompt"] = Category(1, [
+    "Approve your sign-in request", "Your verification code is {code}", "Authenticator re-registration required",
+    "New MFA device pending approval", "Confirm your identity to continue",
+], [
+    ["We sent an approval request to your {device}. Approve it to finish signing in.",
+     "Your authenticator must be re-registered after the {org} security update.",
+     "A verification code ({code}) was requested for your account from {location}.",
+     "Multi-factor authentication for {email} needs to be re-enrolled."],
+    ["Re-register your authenticator here: {url}", "Approve or review the request: {url}",
+     "Complete MFA setup at: {url}", "Confirm your identity: {url}"],
+    ["If you did not request this, approve within {hours} hours to keep access.", "Do not share your code with anyone.", ""],
+    PHISH_MUNDANE,
+], [], ["brand", "itdesk"], anchors=["Approve request", "Re-register authenticator", "Confirm identity", "Review sign-in"])
+
+C["bec_payment"] = Category(1, [
+    "Quick request", "Are you available?", "Urgent - need your help", "Payment to process today",
+    "Confidential - wire request", "Can you handle this now?",
+], [
+    ["Are you at your desk? I need a task handled quickly and I'm heading into {room}.",
+     "I need you to process a payment of {amount} to a vendor before end of day. Keep it between us for now.",
+     "Can you action a wire transfer today? I'll send the banking details once you confirm you're free.",
+     "I'm tied up in meetings - handle this discreetly and let me know when it's done."],
+    ["Reply and I'll send the account details.", "Let me know once it's sent.",
+     "The beneficiary details are attached; confirm the {amount} is cleared today.",
+     "Send me confirmation of the transfer to {email} when complete."],
+    ["Sent from my iPhone, apologies for brevity.", "This is time-sensitive.",
+     "I'm relying on you to keep this quiet until it posts.", ""],
+    ["We're just finishing the planned budget review, so I can't call right now.", "", ""],
+], [], ["ceo"], url="none", html_prob=0.0)
+
+C["bec_giftcard"] = Category(1, [
+    "Quick favor", "Need these today", "Are you around?", "Client appreciation gifts", "Small errand",
+], [
+    ["I need you to pick up {gift_n} {gift_v} gift cards for a client thank-you. Can you do it now?",
+     "Please grab {gift_n} gift cards ({gift_v} each) for the team - I'll reimburse you.",
+     "I'm in back-to-back calls; can you buy gift cards and send me the codes?"],
+    ["Scratch off the back and email me photos of the codes.", "Text me once you have them.",
+     "Send the card numbers to {email} when done."],
+    ["Keep the receipt and I'll sort the expense.", "It's a surprise, so keep it quiet.", ""],
+    ["Just posting this to you rather than the whole team.", "", ""],
+], [], ["ceo"], url="none", html_prob=0.0)
+
+C["fake_fileshare"] = Category(1, [
+    "{col_first} shared \"{file}\" with you", "Document shared via OneDrive", "You have a new shared file",
+    "Please review and sign: {file}", "DocuSign: {file} awaiting your signature", "A file was shared with {email}",
+], [
+    ["{col_first} {col_last} shared the document \"{file}\" with you.",
+     "A file (\"{file}\") has been shared with you via OneDrive for Business.",
+     "{ven_first} {ven_last} sent you \"{file}\" to review and sign.",
+     "You have a DocuSign envelope waiting: \"{file}\"."],
+    ["Open the document here: {url}", "Review and sign: {url}", "View \"{file}\": {url}", "Access the shared file: {url}",
+     "Open the shared folder: {url}"],
+    ["This link is available for {days} days.", "Sign in with your email to view.",
+     "You may be asked to verify your account to open it.", ""],
+    PHISH_MUNDANE,
+], [], ["docs", "hr", "colleague"], anchors=["Open document", "View document", "Review and sign", "Open in OneDrive"])
+
+C["invoice_lure"] = Category(1, [
+    "Overdue invoice {inv}", "Final notice: payment required", "Invoice {inv} past due", "Outstanding balance {amount}",
+    "Payment reminder - {inv}", "Your account is overdue",
+], [
+    ["Invoice {inv} for {amount} is now overdue. Please remit payment to avoid service interruption.",
+     "Our records show an unpaid balance of {amount} on your account. See the attached invoice.",
+     "This is a final notice for overdue invoice {inv}. Immediate payment is required."],
+    ["View and pay the invoice here: {url}", "Open the invoice {inv}: {url}", "Settle the balance online: {url}",
+     "Download the statement: {url}"],
+    ["A late fee applies after {days} days.", "Reference: {inv}.", "Please confirm once paid.",
+     "Your monthly statement will show the charge once it clears.", ""],
+    PHISH_MUNDANE,
+], [], ["vendor", "itdesk"], anchors=["View invoice", "Pay now", "Download statement", "View balance"])
+
+C["helpdesk_quota"] = Category(1, [
+    "Mailbox almost full - {pct}%", "Your email storage is full", "Action needed: mailbox quota exceeded",
+    "Incoming messages are being held", "Re-validate your mailbox", "Email account deactivation notice",
+], [
+    ["Your mailbox is at {gb_used} GB of {gb_total} GB ({pct}%). New mail will bounce once it is full.",
+     "Your email storage for {email} is full and incoming messages are being held.",
+     "Your account is scheduled for deactivation unless it is re-validated under the {org} policy."],
+    ["Increase your quota here: {url}", "Re-validate your mailbox: {url}", "Release held messages: {url}",
+     "Keep your account active: {url}"],
+    ["This request expires in {hours} hours.", "Sign in with your current credentials to continue.", ""],
+    PHISH_MUNDANE,
+], [], ["itdesk", "brand"], anchors=["Increase quota", "Re-validate mailbox", "Release messages", "Keep account active"])
+
+C["vendor_bankchange"] = Category(1, [
+    "Updated remittance details", "Change to our banking information", "New payment instructions - {vendor}",
+    "Please update our bank details", "Important: ACH information change",
+], [
+    ["We have changed banks. Please update our remittance details before your next payment of {amount}.",
+     "Effective immediately, {vendor} payments should go to our new account. Updated details attached.",
+     "Our ACH information has changed for invoice {inv}; please use the new account ending {acct4}."],
+    ["Confirm the updated details here: {url}", "Download the updated banking form: {url}",
+     "Review the new remittance instructions: {url}"],
+    ["Please confirm receipt so future payments are not delayed.", "Apply this to all open invoices.", ""],
+    PHISH_MUNDANE,
+], [], ["vendor"], anchors=["Confirm details", "Download form", "View instructions"])
+
+# ------------------------------------------------------------------ negatives (label 0)
+# Same businesses, same tone, and every category carries a link to a plausible internal / vendor portal
+# ({url} with url="legit"). Benign sentences reuse the lure vocabulary (password, verify, approve,
+# overdue, OneDrive, DocuSign, gift cards, wire, expires, suspended, urgent, authenticator, quota,
+# held messages, envelope, Entra) so those tokens are not label-predictive.
+C["internal_schedule"] = Category(0, [
+    "Staff meeting {date}", "Schedule for next week", "Can we move our 1:1?", "Coverage for {date}",
+    "Reminder: team lunch {date}", "Shift swap for {date}",
+], [
+    ["Can we meet in {room} on {date} at {time} to go over the week?",
+     "Just confirming our 1:1 on {date} at {time} - let me know if you need to move it.",
+     "I'm putting together coverage for {date}; can you take the morning front desk?",
+     "Team lunch is on {date}. Please reply with your order by {date2}."],
+    ["The agenda doc is on OneDrive: {url}", "Shared calendar, if you want to check conflicts: {url}",
+     "I'll book {room}; the room schedule is here: {url}", "No agenda, just a quick sync.", ""],
+    ["Also, could you approve my PTO request in the HR portal when you get a sec?",
+     "If the booking system asks you to sign in, your usual password works - nothing has changed.",
+     "Bring your notes from last week.", "Thanks for being flexible.", ""],
+], [], ["colleague"], url="legit", html_prob=0.1, anchors=["Agenda", "Open doc", "Room calendar"])
+
+C["genuine_invoice"] = Category(0, [
+    "Invoice {inv} from {vendor}", "Your {vendor} statement", "Monthly invoice - {date}", "Receipt for {inv}",
+    "{vendor}: invoice attached", "Statement for {org}",
+], [
+    ["Please find invoice {inv} for {amount2} attached, covering this month's supplies.",
+     "Your statement from {vendor} is ready. The balance of {amount2} is due {date2}.",
+     "Thanks for your order. Invoice {inv} is attached; no action needed if already paid.",
+     "Here is your monthly invoice. You can pay by the usual bank transfer or check."],
+    ["View or pay invoice {inv} in the portal: {url}", "A copy is also in your {vendor} portal: {url}",
+     "Download the PDF from your account: {url}", "Payment history and open invoices: {url}"],
+    ["This is not overdue - the usual {days}-day terms apply.",
+     "Our banking details are unchanged; if you ever get a message saying otherwise, call {phone} to verify before paying.",
+     "If you already sent payment, please disregard this.",
+     "The portal login uses your existing password; there is nothing to re-verify.", ""],
+], [], ["vendor"], url="legit", html_prob=0.25, anchors=["View invoice", "Pay online", "Open portal"])
+
+C["service_health"] = Category(0, [
+    "Microsoft 365 service health update", "Scheduled maintenance notice", "Outlook degradation - resolved",
+    "Service advisory for {date}", "Planned Teams maintenance",
+], [
+    ["Microsoft 365 reported a service issue affecting Outlook on the web; it is now resolved. No action is needed.",
+     "There is a planned maintenance window on {date} from {time}. Some Teams features may be briefly unavailable.",
+     "This is an informational service-health advisory. You do not need to sign in or change anything.",
+     "Entra ID sign-in was slow for about {hours} hours this morning; Microsoft has posted the root cause."],
+    ["Full status: {url}", "Details are on the status page: {url}", "Message center post: {url}",
+     "Track the advisory here: {url}"],
+    ["No sign-in, password reset or authenticator re-registration is required.",
+     "Mailbox quota and held messages were not affected.",
+     "Nobody from IT will ever ask you to approve an MFA request you did not start.",
+     "Posting for awareness only.", ""],
+], [], ["itdesk", "msp"], url="legit", html_prob=0.25, anchors=["Full status", "Status page", "View advisory"])
+
+C["hr_benefits"] = Category(0, [
+    "Open enrollment reminder", "Updated employee handbook", "Payroll calendar for 2026", "Benefits Q&A on {date}",
+    "New PTO policy", "Holiday schedule",
+], [
+    ["Open enrollment closes {date2}. Review your benefits elections and let HR know of any changes.",
+     "The updated employee handbook (\"{file}\") is posted on the intranet for your reference.",
+     "Here is the 2026 payroll calendar. Direct deposit dates are unchanged.",
+     "We're holding a benefits Q&A on {date} at {time} in {room}."],
+    ["Sign the handbook acknowledgement in the DocuSign envelope by {date2}: {url}",
+     "The payroll calendar is in the shared OneDrive folder: {url}",
+     "Enrollment portal (same login as payroll): {url}", "Handbook on the intranet: {url}",
+     "Please confirm your elections on the enrollment form by {date2}."],
+    ["Enrollment access expires {date2}; late elections are held until next year.",
+     "Holiday gift cards are ordered centrally by Accounts Payable - nobody will ask you to buy them.",
+     "No action is needed if your current elections are correct.", "Reach HR at {phone} ext. {ext}.",
+     "It's for all staff, so please pass it along within the week.", ""],
+], [], ["hr"], url="legit", html_prob=0.2, anchors=["Open handbook", "Sign in DocuSign", "Enrollment portal", "View calendar"])
+
+C["project_update"] = Category(0, [
+    "{project} - weekly update", "Status: {project}", "Next steps on the {project}", "{project} recap",
+    "Quick update on the {project}",
+], [
+    ["Quick update on the {project}: we finished the first phase and start the next on {date2}.",
+     "Here's where the {project} stands. We're on track for the {date2} milestone.",
+     "Recap from today's call on the {project} - action items are below.",
+     "The {project} is moving along; I'll need sign-off on the timeline by {date2}."],
+    ["The updated plan (\"{file}\") is in the shared OneDrive folder: {url}", "Timeline and action items: {url}",
+     "The project SharePoint site: {url}", "Ticket history for the {project}: {url}",
+     "I'll share the planning document with the client once it's final."],
+    ["Nothing urgent - this can wait until our call.",
+     "Access to the shared folder was granted today; sign in with your work account.",
+     "I'll need your approval on the revised {amount} budget line before we commit.",
+     "Please confirm the dates work and I'll lock them in.", "No blockers on our end.", ""],
+], [], ["colleague", "msp"], url="legit", html_prob=0.15, anchors=["Updated plan", "Open folder", "View timeline"])
+
+C["vendor_quote"] = Category(0, [
+    "Quote for {project}", "Revised estimate - {vendor}", "Pricing you requested", "Proposal for {org}",
+    "Estimate {inv}",
+], [
+    ["Thanks for the opportunity. Our quote for the {project} comes to {amount}, valid for {days} days.",
+     "Attached is the revised estimate (\"{file}\") for the {project}.",
+     "Here's the pricing you asked about. Happy to adjust the scope if needed.",
+     "Our proposal for {org} is attached; the {amount} total includes materials and labor."],
+    ["View the quote online: {url}", "Approve or request changes here: {url}",
+     "Full estimate (\"{file}\"): {url}", "Our project portfolio, if useful: {url}",
+     "It's attached as a PDF; the signed order form can come back the same way."],
+    ["The quote expires in {days} days; just let us know if you need longer.",
+     "Payment terms are 50% deposit by bank transfer, balance on completion - no wire is needed until you approve.",
+     "Please confirm within {days} days so we can hold the pricing.",
+     "Call {phone} with any questions.", "No obligation, of course.", ""],
+], [], ["vendor", "client"], url="legit", html_prob=0.2, anchors=["View quote", "Open estimate", "Approve quote"])
+
+C["calendar_confirm"] = Category(0, [
+    "Confirmed: {date} at {time}", "Appointment reminder", "Your booking with {org}", "See you {date}",
+    "Reminder: {date} appointment",
+], [
+    ["This confirms your appointment with {org} on {date} at {time}.",
+     "Reminder: you're booked for {date} at {time}. Reply to reschedule.",
+     "Looking forward to seeing you on {date}. Please arrive a few minutes early."],
+    ["Reschedule or cancel online: {url}", "Manage your booking: {url}", "Add it to your calendar: {url}",
+     "Please confirm by replying YES, or call us to reschedule.", ""],
+    ["Your confirmation code is {code} - keep it handy at check-in.",
+     "Your {device} may ask you to approve the calendar invite.",
+     "No action needed - just a reminder.", "We'll send a text the day before.", ""],
+], [], ["colleague"], url="legit", html_prob=0.2, anchors=["Reschedule", "Manage booking", "Add to calendar"])
+
+C["msp_maintenance"] = Category(0, [
+    "Scheduled maintenance window {date}", "After-hours patching {date}", "Firewall update tonight",
+    "Backup verification complete", "Planned reboot {date}",
+], [
+    ["{msp} will perform scheduled maintenance on {date} from {time}. Brief downtime is possible after hours.",
+     "We're applying security patches to your server on {date}. No action is needed on your end.",
+     "Tonight's firewall update may interrupt the VPN for a few minutes.",
+     "This month's backup verification for {org} completed successfully; copy on file."],
+    ["Ticket {ticket}: {url}", "Live status during the window: {url}",
+     "Maintenance notes and affected systems: {url}", "Ticket {ticket} tracks this work.", ""],
+    ["Please verify your laptop rebooted and that you can sign in as usual afterwards.",
+     "If a login prompt appears after the reboot, use your normal password; we will never ask for it by email.",
+     "Authenticator and MFA settings are unchanged.", "Reach the service desk at {phone}.", ""],
+], [], ["msp", "itdesk"], url="legit", html_prob=0.2, anchors=["View ticket", "Status page", "Open ticket"])
+
+POSITIVE = [k for k, v in C.items() if v.label == 1]
+NEGATIVE = [k for k, v in C.items() if v.label == 0]
+
+
+def label3_for_category(cat_name: str) -> int:
+    """Three-way label of a template category: "0:*" -> 0 (legit), "1:*" -> 1 (phish).
+
+    A positive category not listed in PHISH_CATEGORIES raises: a future spam-style template must be
+    classified deliberately (label3 2) instead of inheriting "phishing" from the binary label.
+    """
+    binary = C[cat_name].label
+    if binary == 0:
+        return 0
+    if cat_name not in PHISH_CATEGORIES:
+        raise ValueError(f"positive category {cat_name!r} is not in PHISH_CATEGORIES; decide phishing vs spam")
+    return 1
+
+
+# ----------------------------------------------------------------------------- rendering
+def _fill(s: str, ctx: dict[str, str]) -> str:
+    return s.format(**ctx)
+
+
+def _recase_subject(sub: str, rng: random.Random) -> str:
+    r = rng.random()
+    if r < 0.12:
+        return sub.upper()
+    if r < 0.24:
+        return sub.lower()
+    return sub
+
+
+def _anchor(url: str, rng: random.Random, html: bool, anchors: list[str]) -> str:
+    """HTML bodies get category-appropriate link text; plain-text bodies get the bare URL because the
+    template sentence already carries the call-to-action prefix."""
+    if html:
+        return f'<a href="{url}">{rng.choice(anchors)}</a>'
+    return url
+
+
+def render(cat_name: str, ctx: dict[str, str], rng: random.Random) -> tuple[str, str]:
+    cat = C[cat_name]
+    html = rng.random() < cat.html_prob
+    subject = _recase_subject(_fill(rng.choice(SUBJECT_PREFIX) + rng.choice(cat.subjects), ctx), rng)
+
+    lines: list[str] = []
+    greet = _fill(rng.choice(GREETINGS), ctx)
+    if greet:
+        lines.append(greet)
+
+    url = ctx["phish_url"] if cat.url == "phish" else ctx["legit_url"]
+    body_sents: list[str] = []
+    for slot in cat.parts:
+        choice = rng.choice(slot)
+        if not choice:
+            continue
+        # weave the link in before slot filling so the anchor text (which may contain no braces)
+        # survives str.format; {url} is the only marker not present in ctx.
+        if "{url}" in choice:
+            if cat.url == "none":
+                continue
+            choice = choice.replace("{url}", _anchor(url, rng, html, cat.anchors))
+        body_sents.append(_fill(choice, ctx))
+    lines.append(" ".join(body_sents))
+
+    signoff = _fill(rng.choice(SIGNOFFS), ctx)
+    signature = _fill(rng.choice(SIGNATURES[rng.choice(cat.signers)]), ctx)
+    disclaimer = _fill(rng.choice(DISCLAIMERS), ctx) if rng.random() < 0.3 else ""
+    tail = [p for p in (signoff, signature, disclaimer) if p]
+
+    # Per-row length target: right-skewed triangular over [MIN_WORDS, TARGET_MAX_WORDS] with the mode at
+    # MIN_WORDS, so most rows stay short like real mail but a tail reaches ~200 words. The draw is
+    # label-independent, and the padding comes from pools shared by both labels, so length and filler
+    # vocabulary carry no label signal. The whole prospective message is counted so every body clears
+    # MIN_WORDS.
+    target = int(rng.triangular(MIN_WORDS, TARGET_MAX_WORDS, MIN_WORDS))
+    pad_pool = list(dict.fromkeys(cat.extras + SHARED_FILLER))
+    rng.shuffle(pad_pool)
+    i = 0
+    tail_words = sum(len(p.split()) for p in tail)
+    while len(" ".join(lines).split()) + tail_words < target and i < len(pad_pool):
+        lines.append(_fill(pad_pool[i], ctx))
+        i += 1
+    lines.extend(tail)
+
+    body = "\n\n".join(p for p in lines if p).strip()
+    if html:
+        body = "<html><body>" + body.replace("\n\n", " <br><br> ") + "</body></html>"
+    # trim to the upper word bound without cutting a word
+    words = body.split()
+    if len(words) > MAX_WORDS:
+        body = " ".join(words[:MAX_WORDS])
+    return subject, body
+
+
+# ----------------------------------------------------------------------------- build
+def _token_purity(df: pd.DataFrame) -> dict[str, Any]:
+    """Document-frequency label purity over the WHOLE vocabulary.
+
+    For each lower-cased word token, count how many positive / negative documents contain it (presence,
+    not multiplicity). Purity = max(pos, neg) / support. A high-support, high-purity token is a shortcut
+    the classifier could exploit. The guard flags every token with support >= PURITY_SUPPORT and purity
+    > PURITY_MAX regardless of rank; the top-N table is kept only for display, because a shortcut token
+    confined to one category (support ~ n/16) never reaches the top of the frequency list.
+    """
+    pos_df: Counter[str] = Counter()
+    neg_df: Counter[str] = Counter()
+    for text, label in zip(df["text"], df["label"]):
+        toks = set(TOKEN_RE.findall(text.lower()))
+        (pos_df if label == 1 else neg_df).update(toks)
+    support = pos_df + neg_df
+
+    def row(tok: str) -> dict[str, Any]:
+        p, n = pos_df[tok], neg_df[tok]
+        return {"token": tok, "support": support[tok], "pos": p, "neg": n,
+                "purity": round(max(p, n) / support[tok], 3), "majority": 1 if p >= n else 0}
+
+    top = [row(t) for t, _ in support.most_common(PURITY_TOP_N)]
+    eligible = [t for t, s in support.items() if s >= PURITY_SUPPORT]
+    flagged = sorted((row(t) for t in eligible if max(pos_df[t], neg_df[t]) / support[t] > PURITY_MAX),
+                     key=lambda r: (-r["support"], r["token"]))
+    return {"top": top, "flagged": flagged, "n_tokens": len(support), "n_eligible": len(eligible)}
+
+
+def build(n: int, seed: int) -> tuple[pd.DataFrame, dict[str, Any]]:
+    rng = random.Random(seed)
+    half = n // 2
+    plan: list[str] = []
+    for i in range(half):
+        plan.append(POSITIVE[i % len(POSITIVE)])
+    for i in range(n - half):
+        plan.append(NEGATIVE[i % len(NEGATIVE)])
+    rng.shuffle(plan)
+
+    seen: set[str] = set()
+    records: list[dict[str, Any]] = []
+    attempts = 0
+    idx = 0
+    while idx < len(plan) and attempts < n * 60:
+        attempts += 1
+        cat_name = plan[idx]
+        ctx = make_ctx(rng)
+        subject, body = render(cat_name, ctx, rng)
+        text = f"{subject}\n\n{body}"
+        key = common.normalize_text(text)
+        if not key or key in seen:
+            continue
+        seen.add(key)
+        records.append({
+            "doc_id": f"synthetic:{idx:06d}", "text": text, "subject": subject, "body": body,
+            "label": C[cat_name].label, "source": "synthetic", "cluster_id": -1, "category": cat_name,
+        })
+        idx += 1
+    if idx < len(plan):
+        LOG.warning("only generated %d/%d unique rows before attempt cap", idx, len(plan))
+
+    # label3 / label_set are derived after generation so the RNG stream (and therefore every existing
+    # column) is identical to the binary-era build for the same seed.
+    for rec in records:
+        rec["label3"] = label3_for_category(rec["category"])
+        rec["label_set"] = LABEL_SET_OF[rec["label3"]]
+    df = pd.DataFrame.from_records(records, columns=COLUMNS)
+    df["label3"] = df["label3"].astype("int64")
+    assert ((df["label3"] != 0) == (df["label"] == 1)).all(), "label3 / binary label disagree"
+    return df, _token_purity(df)
+
+
+def _length_stats(df: pd.DataFrame) -> dict[str, Any]:
+    wl = df["body"].str.split().map(len)
+    out: dict[str, Any] = {"min": int(wl.min()), "max": int(wl.max()), "mean": round(float(wl.mean()), 1),
+                           "median": int(wl.median())}
+    for label in (1, 0):
+        w = wl[df["label"] == label]
+        out[f"label_{label}"] = {"min": int(w.min()), "max": int(w.max()), "mean": round(float(w.mean()), 1),
+                                 "p10": int(w.quantile(0.1)), "p90": int(w.quantile(0.9))}
+    return out
+
+
+def _url_stats(df: pd.DataFrame) -> dict[str, Any]:
+    """Presence of a URL / HTML anchor / HTML wrapper per label; these must be close to balanced or the
+    supplement teaches 'link => phishing'."""
+    out: dict[str, Any] = {}
+    for key, pat in (("url", "://"), ("href", "<a href"), ("html", "<html>")):
+        hit = df["text"].str.contains(pat, regex=False)
+        out[key] = {"pos": int(hit[df["label"] == 1].sum()), "neg": int(hit[df["label"] == 0].sum())}
+    return out
+
+
+def _write_parquet(df: pd.DataFrame, out: Path) -> None:
+    table = pa.Table.from_pandas(df, preserve_index=False)
+    md = {**(table.schema.metadata or {}), **{k.encode(): v.encode() for k, v in PARQUET_META.items()}}
+    table = table.replace_schema_metadata(md)
+    out.parent.mkdir(parents=True, exist_ok=True)
+    pq.write_table(table, out)
+
+
+def cmd_build(args: argparse.Namespace, cfg: dict) -> None:
+    cap = int(cfg["data"]["synthetic"]["max_rows"])
+    n = args.n if args.n is not None else cap
+    if n > cap:
+        LOG.warning("requested n=%d exceeds config max_rows=%d; clamping", n, cap)
+        n = cap
+    out = Path(args.out) if args.out else Path(cfg["paths"]["data_processed"]) / "synthetic.parquet"
+    if not out.is_absolute():
+        out = common.project_root() / out
+    seed = args.seed if args.seed is not None else int(cfg["seed"])
+
+    df, purity = build(n, seed)
+    _write_parquet(df, out)
+
+    dup_rate = 1.0 - df["text"].map(common.normalize_text).nunique() / len(df)
+    by_cat = df.groupby(["label", "category"]).size().reset_index(name="n")
+    counts = {
+        "rows": len(df), "label_1": int((df["label"] == 1).sum()), "label_0": int((df["label"] == 0).sum()),
+        "per_category": {f"{int(r.label)}:{r.category}": int(r.n) for r in by_cat.itertuples()},
+        # Three-way counts (all three classes listed so a reader sees spam = 0 explicitly).
+        "label3": {LABEL3_NAMES[k]: int((df["label3"] == k).sum()) for k in sorted(LABEL3_NAMES)},
+        "label_set": {v: int((df["label_set"] == v).sum()) for v in ("legit", "phish", "spam", "unwanted")},
+        "per_category_label3": {f"{int(r.label)}:{r.category}": label3_for_category(r.category)
+                                for r in by_cat.itertuples()},
+    }
+    url_stats = _url_stats(df)
+    report = {
+        "generator": "make_synthetic.py", "note": PARQUET_META["note"], "seed": seed, "requested_n": n,
+        "output": str(out), "duplicate_rate": round(float(dup_rate), 6), "counts": counts,
+        "length_words": _length_stats(df), "url_presence": url_stats,
+        "purity_support_threshold": PURITY_SUPPORT, "purity_warn_above": PURITY_MAX,
+        "vocab_tokens": purity["n_tokens"], "tokens_with_support_ge_threshold": purity["n_eligible"],
+        "token_purity_top30": purity["top"],
+        "token_purity_flagged_count": len(purity["flagged"]), "token_purity_flagged": purity["flagged"],
+    }
+    report_path = out.with_name(out.stem + "_report.json")
+    common.write_json(report, report_path)
+
+    LOG.info("wrote %d rows (%d pos / %d neg) -> %s", counts["rows"], counts["label_1"], counts["label_0"], out)
+    LOG.info("three-way: %s (label_set %s)", counts["label3"], counts["label_set"])
+    LOG.info("duplicate rate (normalize_text): %.6f", dup_rate)
+    LOG.info("length words: %s", report["length_words"])
+    LOG.info("url presence (pos/neg): %s", url_stats)
+    LOG.info("report -> %s", report_path)
+
+    LOG.info("per-token label purity (top %d by support; support = #docs containing the token):", len(purity["top"]))
+    LOG.info("%-16s%8s%7s%7s%8s", "token", "support", "pos", "neg", "purity")
+    for r in purity["top"]:
+        LOG.info("%-16s%8d%7d%7d%8.3f", r["token"], r["support"], r["pos"], r["neg"], r["purity"])
+    flagged = purity["flagged"]
+    LOG.info("purity scan over full vocabulary: %d tokens, %d with support>=%d",
+             purity["n_tokens"], purity["n_eligible"], PURITY_SUPPORT)
+    if flagged:
+        LOG.warning("%d token(s) with support>=%d are >%.0f%% one class (possible shortcut):",
+                    len(flagged), PURITY_SUPPORT, PURITY_MAX * 100)
+        for r in flagged:
+            LOG.warning("  %-16s support=%4d pos=%4d neg=%4d purity=%.3f majority=%d",
+                        r["token"], r["support"], r["pos"], r["neg"], r["purity"], r["majority"])
+    else:
+        LOG.info("no token with support>=%d exceeds %.0f%% one class.", PURITY_SUPPORT, PURITY_MAX * 100)
+
+
+def cmd_show(args: argparse.Namespace, cfg: dict) -> None:
+    path = Path(args.path) if args.path else Path(cfg["paths"]["data_processed"]) / "synthetic.parquet"
+    if not path.is_absolute():
+        path = common.project_root() / path
+    df = pd.read_parquet(path)
+    for label in (1, 0):
+        sub = df[df["label"] == label].head(args.k)
+        print(f"\n{'=' * 78}\nlabel={label} ({'phishing/spam' if label else 'legitimate'}) - {len(sub)} samples")
+        for row in sub.itertuples():
+            l3 = getattr(row, "label3", None)
+            three = f"  label3={l3} ({LABEL3_NAMES.get(l3, '?')}) label_set={getattr(row, 'label_set', '?')}" \
+                if l3 is not None else "  label3=<missing: rebuild>"
+            print(f"\n--- {row.doc_id}  [{row.category}]  source={row.source} cluster_id={row.cluster_id}{three}")
+            print(f"SUBJECT: {row.subject}")
+            print(row.body)
+
+
+def main(argv: list[str] | None = None) -> None:
+    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
+    ap.add_argument("--config", default="config.yaml")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    b = sub.add_parser("build", help="write the synthetic parquet + report json")
+    b.add_argument("--n", type=int, default=None, help="rows to generate (default: config data.synthetic.max_rows)")
+    b.add_argument("--out", default=None, help="output parquet path (default: data/processed/synthetic.parquet)")
+    b.add_argument("--seed", type=int, default=None, help="override config seed")
+    s = sub.add_parser("show", help="print k sample rows per class")
+    s.add_argument("--k", type=int, default=6)
+    s.add_argument("--path", default=None, help="parquet to read (default: data/processed/synthetic.parquet)")
+    args = ap.parse_args(argv)
+
+    cfg = common.load_config(args.config)
+    global LOG
+    LOG = common.setup_logging("make_synthetic", cfg["paths"]["logs"])
+    {"build": cmd_build, "show": cmd_show}[args.cmd](args, cfg)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/presentation/src/model.py b/presentation/src/model.py
new file mode 100644
index 0000000000000000000000000000000000000000..cc07f2f4c77390f7a9d0e48a3c46f7ae1c543e63
--- /dev/null
+++ b/presentation/src/model.py
@@ -0,0 +1,524 @@
+"""From-scratch pre-LayerNorm BERT-style encoder for the BITS Phishing Detector.
+
+Plain PyTorch only (torch / math / dataclasses / typing) so this file can be copied
+verbatim to app/model.py for the Hugging Face Space. No pretrained weights, no
+``transformers``.
+
+Architecture (docs/research/research_model-math.md sec. 1 and 5):
+
+* token + learned absolute position embeddings -> LayerNorm -> dropout (no segment
+  embeddings: the classifier sees a single "subject\\n\\nbody" segment).
+* ``n_layers`` pre-LN blocks::
+
+      x = x + Dropout(OutProj(SDPA(QKV(LN1(x)))))
+      x = x + Dropout(FFN_down(GELU(FFN_up(LN2(x)))))
+
+  Pre-LN is chosen over BERT's post-LN because a 24-layer stack trained from
+  scratch at batch 256 is far more tolerant of the learning rate and warmup under
+  pre-LN (Xiong et al. 2020), and we have no pretrained weights to fall back on.
+* a final LayerNorm (pre-LN needs it: the residual stream is otherwise un-normalised).
+* attention through ``torch.nn.functional.scaled_dot_product_attention`` so the
+  fused Flash / memory-efficient kernels run and the L x L score matrix is never
+  stored for backward. No mask -> ``attn_mask=None`` (Flash-eligible, the packed
+  pretraining path); a ``BoolTensor[B, L]`` key-padding mask (True = keep) ->
+  ``[B, 1, 1, L]`` (memory-efficient backend, the fine-tune / inference path).
+
+Default config (24L / d1024 / 16 heads / FFN 4096 / V 32768 / L 512) has exactly
+337,476,608 parameters with the tied MLM head and 337,443,842 as the classifier.
+"""
+
+import math
+from dataclasses import asdict, dataclass, fields
+from typing import Any, Mapping
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from torch import Tensor
+from torch.utils.checkpoint import checkpoint
+
+
+@dataclass
+class EncoderConfig:
+    vocab_size: int = 32768
+    max_seq_len: int = 512
+    d_model: int = 1024
+    n_layers: int = 24
+    n_heads: int = 16
+    d_ff: int = 4096
+    dropout: float = 0.1
+    attention_dropout: float = 0.1
+    # 1e-5 rather than BERT's 1e-12. Under autocast LayerNorm runs in fp32 so both
+    # behave identically in training; 1e-5 is the safer value if the deployed model
+    # is ever run in pure bf16 (8-bit mantissa: var + 1e-12 == var), and it is the
+    # value every modern pre-LN encoder (RoBERTa-style, GPT-2) uses.
+    layer_norm_eps: float = 1e-5
+    pad_token_id: int = 0
+    tie_mlm_head: bool = True
+    grad_checkpointing: bool = False
+    num_labels: int = 2
+
+    def __post_init__(self) -> None:
+        if self.d_model % self.n_heads:
+            raise ValueError(f"d_model={self.d_model} not divisible by n_heads={self.n_heads}")
+        if not 0 <= self.pad_token_id < self.vocab_size:
+            raise ValueError("pad_token_id outside the vocabulary")
+
+    @property
+    def head_dim(self) -> int:
+        return self.d_model // self.n_heads
+
+    @classmethod
+    def from_dict(cls, d: Mapping[str, Any]) -> "EncoderConfig":
+        """Build from the config.yaml ``model`` section.
+
+        Keys that describe choices this file implements only one way (``norm``,
+        ``activation``) are validated rather than silently ignored; any other
+        unknown key is dropped.
+        """
+        norm = d.get("norm", "pre_ln")
+        if norm != "pre_ln":
+            raise ValueError(f"only norm=pre_ln is implemented, got {norm!r}")
+        act = d.get("activation", "gelu")
+        if act != "gelu":
+            raise ValueError(f"only activation=gelu is implemented, got {act!r}")
+        known = {f.name for f in fields(cls)}
+        return cls(**{k: v for k, v in d.items() if k in known})
+
+    def to_dict(self) -> dict[str, Any]:
+        return asdict(self)
+
+
+# --------------------------------------------------------------------------- init
+def _init_module(module: nn.Module) -> None:
+    """BERT init: N(0, 0.02) weights, zero biases, identity LayerNorm."""
+    if isinstance(module, nn.Linear):
+        nn.init.normal_(module.weight, mean=0.0, std=0.02)
+        if module.bias is not None:
+            nn.init.zeros_(module.bias)
+    elif isinstance(module, nn.Embedding):
+        nn.init.normal_(module.weight, mean=0.0, std=0.02)
+        if module.padding_idx is not None:
+            with torch.no_grad():
+                module.weight[module.padding_idx].zero_()
+    elif isinstance(module, nn.LayerNorm):
+        nn.init.ones_(module.weight)
+        nn.init.zeros_(module.bias)
+
+
+def _zero_pad_row_grad(pad_token_id: int):
+    """Post-accumulate hook keeping the [PAD] embedding row exactly zero through training.
+
+    ``nn.Embedding(padding_idx)`` routes no gradient into that row, but with the tied MLM
+    decoder the output matmul does (observed grad norm ~0.06 after one backward), so
+    the row would drift from zero on the first optimizer step. Zeroing ``.grad[pad]``
+    after accumulation is enough: AdamW then has zero moments for the row and decoupled
+    weight decay leaves a zero weight at zero.
+    """
+    def hook(param: Tensor) -> None:
+        if param.grad is not None:
+            param.grad[pad_token_id].zero_()
+    return hook
+
+
+def _expand_key_padding_mask(attention_mask: Tensor | None) -> Tensor | None:
+    """``BoolTensor[B, L]`` (True = keep) -> ``[B, 1, 1, L]`` broadcast over heads and queries."""
+    if attention_mask is None:
+        return None
+    if attention_mask.dtype is not torch.bool:
+        attention_mask = attention_mask.bool()
+    return attention_mask[:, None, None, :]
+
+
+# ------------------------------------------------------------------------ modules
+class Embeddings(nn.Module):
+    """Token + learned absolute position embeddings -> LayerNorm -> dropout.
+
+    The [PAD] row of the token table is zero at init and stays zero while training
+    (see ``_zero_pad_row_grad``). Tensor hooks are not carried by ``state_dict`` or
+    ``copy.deepcopy``: a deep-copied module must not be trained, a module rebuilt from
+    its config and ``load_state_dict`` is fine.
+    """
+
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.tok = nn.Embedding(cfg.vocab_size, cfg.d_model, padding_idx=cfg.pad_token_id)
+        self.pos = nn.Embedding(cfg.max_seq_len, cfg.d_model)
+        self.norm = nn.LayerNorm(cfg.d_model, eps=cfg.layer_norm_eps)
+        self.dropout = nn.Dropout(cfg.dropout)
+        self.register_buffer("position_ids", torch.arange(cfg.max_seq_len), persistent=False)
+        self.tok.weight.register_post_accumulate_grad_hook(_zero_pad_row_grad(cfg.pad_token_id))
+
+    def forward(self, input_ids: Tensor) -> Tensor:
+        L = input_ids.shape[1]
+        if L > self.pos.num_embeddings:
+            raise ValueError(f"sequence length {L} > max_seq_len {self.pos.num_embeddings}")
+        x = self.tok(input_ids) + self.pos(self.position_ids[:L])
+        return self.dropout(self.norm(x))
+
+
+class EncoderLayer(nn.Module):
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.n_heads = cfg.n_heads
+        self.head_dim = cfg.head_dim
+        self.attention_dropout = cfg.attention_dropout
+        self.ln1 = nn.LayerNorm(cfg.d_model, eps=cfg.layer_norm_eps)
+        self.qkv = nn.Linear(cfg.d_model, 3 * cfg.d_model)
+        self.out_proj = nn.Linear(cfg.d_model, cfg.d_model)
+        self.ln2 = nn.LayerNorm(cfg.d_model, eps=cfg.layer_norm_eps)
+        self.ffn_up = nn.Linear(cfg.d_model, cfg.d_ff)
+        self.ffn_down = nn.Linear(cfg.d_ff, cfg.d_model)
+        self.dropout = nn.Dropout(cfg.dropout)
+
+    def _qkv(self, x: Tensor) -> tuple[Tensor, Tensor, Tensor]:
+        B, L, _ = x.shape
+        qkv = self.qkv(x).view(B, L, 3, self.n_heads, self.head_dim)
+        q, k, v = qkv.unbind(dim=2)
+        return q.transpose(1, 2), k.transpose(1, 2), v.transpose(1, 2)  # [B, h, L, dh]
+
+    def forward(self, x: Tensor, attn_mask: Tensor | None = None) -> Tensor:
+        q, k, v = self._qkv(self.ln1(x))
+        o = F.scaled_dot_product_attention(
+            q, k, v,
+            attn_mask=attn_mask,
+            dropout_p=self.attention_dropout if self.training else 0.0,
+        )
+        o = o.transpose(1, 2).reshape(x.shape)
+        x = x + self.dropout(self.out_proj(o))
+        x = x + self.dropout(self.ffn_down(F.gelu(self.ffn_up(self.ln2(x)))))
+        return x
+
+    @torch.no_grad()
+    def attention_probs(self, x: Tensor, attn_mask: Tensor | None = None) -> Tensor:
+        """Explicit softmax(QK^T / sqrt(dh)) for inspection. Off the hot path.
+
+        fp32, no dropout: this is the quantity the app shows as a signal, not the
+        stochastic one used in training. Returns ``[B, h, L, L]``.
+        """
+        q, k, _ = self._qkv(self.ln1(x))
+        scores = (q.float() @ k.float().transpose(-1, -2)) / math.sqrt(self.head_dim)
+        if attn_mask is not None:
+            scores = scores.masked_fill(~attn_mask, float("-inf"))
+        return scores.softmax(dim=-1)
+
+
+class Encoder(nn.Module):
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.config = cfg
+        self.embeddings = Embeddings(cfg)
+        self.layers = nn.ModuleList(EncoderLayer(cfg) for _ in range(cfg.n_layers))
+        self.final_norm = nn.LayerNorm(cfg.d_model, eps=cfg.layer_norm_eps)
+        self.apply(_init_module)
+        self._scale_residual_projections()
+
+    def _scale_residual_projections(self) -> None:
+        """GPT-2 style 1/sqrt(2 * n_layers) on the two projections that write into the
+        residual stream (OutProj, FFN_down).
+
+        Each of the 2 * n_layers residual branches adds an N(0, 0.02^2)-scaled
+        contribution; without the rescale the residual variance grows linearly with
+        depth, so the final-norm input and the early-training gradient magnitude
+        would be ~sqrt(48) ~ 7x larger for 24 layers than for one. Keeping the stream
+        O(1) at init is what lets a from-scratch 24-layer pre-LN model take a 1.5e-4
+        peak LR with a short warmup instead of diverging in the first few hundred steps.
+        """
+        scale = 1.0 / math.sqrt(2 * self.config.n_layers)
+        with torch.no_grad():
+            for layer in self.layers:
+                layer.out_proj.weight.mul_(scale)
+                layer.ffn_down.weight.mul_(scale)
+
+    def forward(
+        self,
+        input_ids: Tensor,
+        attention_mask: Tensor | None = None,
+        return_last_attn: bool = False,
+    ) -> Tensor | tuple[Tensor, Tensor]:
+        mask = _expand_key_padding_mask(attention_mask)
+        x = self.embeddings(input_ids)
+        use_ckpt = self.config.grad_checkpointing and self.training and torch.is_grad_enabled()
+        last_attn: Tensor | None = None
+        n = len(self.layers)
+        for i, layer in enumerate(self.layers):
+            if return_last_attn and i == n - 1:
+                last_attn = layer.attention_probs(x, mask)
+            if use_ckpt:
+                x = checkpoint(layer, x, mask, use_reentrant=False)
+            else:
+                x = layer(x, mask)
+        x = self.final_norm(x)
+        if return_last_attn:
+            assert last_attn is not None
+            return x, last_attn
+        return x
+
+
+class MLMHead(nn.Module):
+    """Linear(d, d) -> GELU -> LayerNorm -> decoder Linear(d, V) tied to the token embedding.
+
+    Tying shares the Parameter object, so ``state_dict()`` carries two keys over one
+    storage (``encoder.embeddings.tok.weight`` and ``head.decoder.weight``) and
+    ``safetensors.torch.save_file`` refuses it ("Some tensors share memory"). Write
+    checkpoints from ``EncoderForMLM.state_dict_for_safetensors()`` (or with
+    ``safetensors.torch.save_model``, which drops the same alias) and read them back
+    with ``EncoderForMLM.load_tied_state_dict``.
+    """
+
+    def __init__(self, cfg: EncoderConfig, token_embedding: nn.Embedding | None) -> None:
+        super().__init__()
+        self.dense = nn.Linear(cfg.d_model, cfg.d_model)
+        self.norm = nn.LayerNorm(cfg.d_model, eps=cfg.layer_norm_eps)
+        self.decoder = nn.Linear(cfg.d_model, cfg.vocab_size, bias=False)
+        # Own bias so the tied matrix is not forced to also fit unigram frequencies.
+        self.bias = nn.Parameter(torch.zeros(cfg.vocab_size))
+        self.apply(_init_module)
+        if token_embedding is not None:
+            # Same Parameter object: one set of weights, one gradient accumulator.
+            # Tying saves 33.5M parameters and (BERT/RoBERTa) regularises the
+            # output distribution toward the input geometry.
+            self.decoder.weight = token_embedding.weight
+
+    def forward(self, hidden: Tensor) -> Tensor:
+        h = self.norm(F.gelu(self.dense(hidden)))
+        return self.decoder(h) + self.bias
+
+
+class EncoderForMLM(nn.Module):
+    """Encoder + MLM head for pretraining. Checkpoints: see ``state_dict_for_safetensors``."""
+
+    TIED_KEY = "head.decoder.weight"
+    EMBEDDING_KEY = "encoder.embeddings.tok.weight"
+
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.config = cfg
+        self.encoder = Encoder(cfg)
+        self.head = MLMHead(cfg, self.encoder.embeddings.tok if cfg.tie_mlm_head else None)
+
+    def state_dict_for_safetensors(self) -> dict[str, Tensor]:
+        """``state_dict()`` minus the tied decoder's alias key.
+
+        With ``tie_mlm_head`` the decoder weight IS the token embedding, and safetensors
+        rejects two keys over one storage, so the pretraining trainer must save this
+        dict (``safetensors.torch.save_file(model.state_dict_for_safetensors(), path)``)
+        rather than the raw ``state_dict()``. ``from_pretrained_encoder`` reads only the
+        ``encoder.*`` keys and accepts either form.
+        """
+        sd = self.state_dict()
+        if self.config.tie_mlm_head:
+            sd.pop(self.TIED_KEY)
+        return sd
+
+    def load_tied_state_dict(self, state_dict: Mapping[str, Tensor], strict: bool = True) -> Any:
+        """``load_state_dict`` that accepts the dedup'd dict from ``state_dict_for_safetensors``
+        / ``save_model``: the dropped alias is re-pointed at the embedding tensor before
+        loading, so strict mode still reports every genuine mismatch."""
+        sd = dict(state_dict)
+        if self.config.tie_mlm_head and self.TIED_KEY not in sd and self.EMBEDDING_KEY in sd:
+            sd[self.TIED_KEY] = sd[self.EMBEDDING_KEY]
+        return self.load_state_dict(sd, strict=strict)
+
+    def forward(
+        self,
+        input_ids: Tensor,
+        attention_mask: Tensor | None = None,
+        masked_positions: Tensor | None = None,
+    ) -> Tensor:
+        """Logits over the vocabulary.
+
+        With ``masked_positions`` (``BoolTensor[B, L]``) the hidden states are
+        gathered BEFORE the head, so logits are ``[N_masked, V]`` and the 85% of
+        positions that carry no loss never materialise a V-wide row (saves ~136 MiB
+        per 512-token sample at peak; research_model-math.md sec. 2).
+        Without it, ``[B, L, V]``.
+        """
+        hidden = self.encoder(input_ids, attention_mask)
+        if masked_positions is not None:
+            hidden = hidden[masked_positions]
+        return self.head(hidden)
+
+
+class ClassificationHead(nn.Module):
+    """CLS -> dropout -> Linear(d, d) -> tanh -> dropout -> Linear(d, num_labels)."""
+
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.dropout = nn.Dropout(cfg.dropout)
+        self.dense = nn.Linear(cfg.d_model, cfg.d_model)
+        self.out = nn.Linear(cfg.d_model, cfg.num_labels)
+        self.apply(_init_module)
+
+    def forward(self, cls_hidden: Tensor) -> Tensor:
+        h = torch.tanh(self.dense(self.dropout(cls_hidden)))
+        return self.out(self.dropout(h))
+
+
+class EncoderForClassification(nn.Module):
+    def __init__(self, cfg: EncoderConfig) -> None:
+        super().__init__()
+        self.config = cfg
+        self.encoder = Encoder(cfg)
+        self.classifier = ClassificationHead(cfg)
+
+    def forward(
+        self,
+        input_ids: Tensor,
+        attention_mask: Tensor | None = None,
+        return_last_attn: bool = False,
+    ) -> Tensor | tuple[Tensor, Tensor]:
+        out = self.encoder(input_ids, attention_mask, return_last_attn=return_last_attn)
+        if return_last_attn:
+            hidden, last_attn = out
+            return self.classifier(hidden[:, 0]), last_attn
+        return self.classifier(out[:, 0])
+
+    @classmethod
+    def from_pretrained_encoder(
+        cls,
+        mlm: "EncoderForMLM | Mapping[str, Tensor]",
+        cfg: EncoderConfig | None = None,
+        num_labels: int | None = None,
+    ) -> "EncoderForClassification":
+        """Classifier whose encoder is initialised from a pretrained ``EncoderForMLM``
+        (module or its ``state_dict``). The MLM head is dropped; the classification
+        head keeps its fresh init. Strict key matching: any missing/unexpected encoder
+        key raises, because a silently half-loaded encoder would train but be garbage.
+        """
+        if isinstance(mlm, EncoderForMLM):
+            cfg = cfg or mlm.config
+            state = mlm.state_dict()
+        else:
+            if cfg is None:
+                raise ValueError("cfg is required when loading from a state_dict")
+            state = mlm
+        if num_labels is not None and num_labels != cfg.num_labels:
+            cfg = EncoderConfig(**{**cfg.to_dict(), "num_labels": num_labels})
+        model = cls(cfg)
+        load_encoder_weights(model.encoder, state, prefix="encoder.", strict=True)
+        return model
+
+
+def load_encoder_weights(
+    encoder: Encoder,
+    state_dict: Mapping[str, Tensor],
+    prefix: str = "encoder.",
+    strict: bool = True,
+) -> tuple[list[str], list[str]]:
+    """Load ``Encoder`` weights from a wrapper's state_dict (keys ``encoder.*``).
+
+    Returns ``(missing_keys, unexpected_keys)`` relative to the encoder. Keys outside
+    ``prefix`` (``head.*``, ``classifier.*``) are ignored, not reported. With
+    ``strict`` any mismatch raises after reporting the exact keys.
+    """
+    sub = {k[len(prefix):]: v for k, v in state_dict.items() if k.startswith(prefix)}
+    result = encoder.load_state_dict(sub, strict=False)
+    missing, unexpected = list(result.missing_keys), list(result.unexpected_keys)
+    if strict and (missing or unexpected):
+        raise RuntimeError(
+            f"encoder weights do not match: missing={missing[:8]}{'...' if len(missing) > 8 else ''} "
+            f"unexpected={unexpected[:8]}{'...' if len(unexpected) > 8 else ''}"
+        )
+    return missing, unexpected
+
+
+def count_parameters(module: nn.Module) -> dict[str, Any]:
+    """Unique-parameter counts (a tied weight is counted once, under the encoder's
+    embedding where ``named_parameters`` first sees it).
+
+    Breakdown keys: ``embeddings`` (token + position + embedding LN),
+    ``encoder_layers``, ``final_norm``, ``head`` (MLM head or classifier).
+    """
+    total = sum(p.numel() for p in module.parameters())
+    trainable = sum(p.numel() for p in module.parameters() if p.requires_grad)
+    breakdown = {"embeddings": 0, "encoder_layers": 0, "final_norm": 0, "head": 0}
+    for name, p in module.named_parameters():
+        parts = name.split(".")
+        if "embeddings" in parts:
+            breakdown["embeddings"] += p.numel()
+        elif "layers" in parts:
+            breakdown["encoder_layers"] += p.numel()
+        elif "final_norm" in parts:
+            breakdown["final_norm"] += p.numel()
+        else:
+            breakdown["head"] += p.numel()
+    return {"total": total, "trainable": trainable, **breakdown}
+
+
+def _format_count_table(title: str, counts: Mapping[str, int]) -> str:
+    rows = [f"{title}"]
+    for k in ("embeddings", "encoder_layers", "final_norm", "head", "total", "trainable"):
+        rows.append(f"  {k:<15} {counts[k]:>15,}")
+    return "\n".join(rows)
+
+
+if __name__ == "__main__":
+    import sys
+    from pathlib import Path
+
+    import yaml  # only here: the module body must stay torch-only for app/model.py
+
+    cfg_path = Path(__file__).resolve().parent.parent / "config.yaml"
+    with open(cfg_path) as fh:
+        model_cfg = EncoderConfig.from_dict(yaml.safe_load(fh)["model"])
+    print(f"config: {model_cfg.to_dict()}")
+
+    mlm = EncoderForMLM(model_cfg)
+    mlm_counts = count_parameters(mlm)
+    print(_format_count_table("EncoderForMLM (tied head)", mlm_counts))
+    assert mlm.head.decoder.weight is mlm.encoder.embeddings.tok.weight, "MLM head is not tied"
+
+    cls = EncoderForClassification.from_pretrained_encoder(mlm)
+    cls_counts = count_parameters(cls)
+    print(_format_count_table("EncoderForClassification", cls_counts))
+
+    is_default = (
+        model_cfg.n_layers, model_cfg.d_model, model_cfg.n_heads, model_cfg.d_ff,
+        model_cfg.vocab_size, model_cfg.max_seq_len, model_cfg.tie_mlm_head, model_cfg.num_labels,
+    ) == (24, 1024, 16, 4096, 32768, 512, True, 2)
+    if is_default:
+        # research_model-math.md sec. 1: per layer 12,596,224; encoder 302,311,424;
+        # embeddings 34,080,768; tied MLM head 1,084,416; classifier head 1,051,650.
+        assert mlm_counts["total"] == 337_476_608, mlm_counts["total"]
+        assert cls_counts["total"] == 337_443_842, cls_counts["total"]
+        assert mlm_counts["encoder_layers"] == 302_309_376, mlm_counts["encoder_layers"]
+        assert mlm_counts["embeddings"] == 34_080_768, mlm_counts["embeddings"]
+        assert mlm_counts["head"] == 1_084_416, mlm_counts["head"]
+        assert cls_counts["head"] == 1_051_650, cls_counts["head"]
+        print("parameter counts match research_model-math.md sec. 1")
+    else:
+        print("non-default config: skipping the 337,476,608 / 337,443,842 assertion")
+
+    # Checkpoint format: the tied alias is dropped for safetensors and restored on load.
+    sd = mlm.state_dict_for_safetensors()
+    assert EncoderForMLM.TIED_KEY not in sd and len(sd) == len(mlm.state_dict()) - 1
+    mlm.load_tied_state_dict(sd, strict=True)
+    EncoderForClassification.from_pretrained_encoder(sd, model_cfg)
+    print(f"safetensors-safe state_dict: {len(sd)} tensors, strict round-trip OK")
+
+    # The [PAD] embedding row must receive no gradient despite the tied decoder.
+    ids = torch.randint(5, model_cfg.vocab_size, (2, 16))
+    mlm.train()
+    mlm(ids).float().mean().backward()
+    pad_grad = mlm.encoder.embeddings.tok.weight.grad[model_cfg.pad_token_id]
+    assert float(pad_grad.abs().max()) == 0.0, "pad row received gradient"
+    assert float(mlm.encoder.embeddings.tok.weight.grad[5:].abs().max()) > 0.0
+    mlm.zero_grad(set_to_none=True)
+    print("pad-row gradient is zero under the tied decoder")
+
+    # Round-trip check of the shared contract on a tiny CPU batch.
+    mlm.eval()
+    mask = torch.ones(2, 16, dtype=torch.bool)
+    mask[1, 10:] = False
+    with torch.inference_mode():
+        full = mlm(ids, mask)
+        pos = torch.zeros(2, 16, dtype=torch.bool)
+        pos[0, 3] = pos[1, 7] = True
+        gathered = mlm(ids, mask, masked_positions=pos)
+        logits, attn = cls(ids, mask, return_last_attn=True)
+    assert full.shape == (2, 16, model_cfg.vocab_size) and gathered.shape == (2, model_cfg.vocab_size)
+    assert logits.shape == (2, model_cfg.num_labels) and attn.shape == (2, model_cfg.n_heads, 16, 16)
+    assert torch.allclose(attn[1, :, :, 10:], torch.zeros(()), atol=0), "padded keys received attention"
+    print("forward shapes OK:", tuple(full.shape), tuple(gathered.shape), tuple(logits.shape), tuple(attn.shape))
+    sys.exit(0)
diff --git a/presentation/src/pretrain.py b/presentation/src/pretrain.py
new file mode 100644
index 0000000000000000000000000000000000000000..67f5b78ec6358bde7cbfb1369a1054df5e89c0c1
--- /dev/null
+++ b/presentation/src/pretrain.py
@@ -0,0 +1,1052 @@
+"""Phase 1: masked-language-model pretraining of the from-scratch encoder (resumable, bf16, VRAM-logged).
+
+This is the long GPU run Nathan launches in tmux, so every choice below favours robustness:
+
+* **Data.** pretrain.parquet is tokenised ONCE into a cached uint16 token stream
+  (data/processed/pretrain_tokens.npy, [SEP] appended after every document exactly like
+  smoke_test's packing) plus a JSON sidecar holding the tokenizer sha1, row counts and the
+  pretrain-val split. The LAST 1% of documents by parquet order is held out as
+  pretrain_val_tokens.npy and is used only for the val MLM loss. The cache is reused when the
+  sidecar's tokenizer sha1 and parquet row count match; otherwise it is rebuilt and the reason
+  is logged. Training sequences are contiguous 512-token windows over the stream (RoBERTa
+  FULL-SENTENCES: text crosses document boundaries, no padding, no attention mask, so SDPA
+  takes the Flash path). Each epoch visits every window once in a permutation seeded with
+  ``seed + epoch``; the data position is therefore (epoch, sequences consumed) and resume
+  reproduces the order exactly.
+* **Optimisation = the measured smoke-test path.** ``mlm_mask`` and ``build_optimizer`` are
+  imported from smoke_test.py (bf16 autocast, fused AdamW with biases/LayerNorm excluded from
+  weight decay, dynamic 15% masking with 80/10/10 replacement, loss on masked positions only
+  with the hidden states gathered before the MLM head, grad-norm clipping). Gradient
+  accumulation is ``round(effective_batch_sequences / micro_batch)`` and the realised batch
+  is logged (a warning when it is not exactly 256, e.g. 24 x 11 = 264).
+* **Schedule.** Linear warmup over ``min(config.warmup_steps, 6% of total steps)`` then cosine
+  (or linear) decay to 10% of the peak over ``epochs x steps_per_epoch`` optimizer steps. The
+  schedule is a pure function of the step and is stored in every checkpoint; a resume must
+  reproduce it exactly (``--max-steps`` / ``--max-hours`` cap how far THIS invocation runs,
+  they never change the plan).
+* **Checkpoints.** ``checkpoints/pretrain/<run>/step-NNNNNNN/{model.safetensors,
+  trainer_state.pt, summary.json}`` written atomically (tmp dir + rename) every
+  ``checkpoint_every_steps``, at every epoch end, on SIGINT/SIGTERM (saved at the next
+  optimizer-step boundary, exit 0), when ``--max-hours`` is reached (exit 0) and when the
+  VRAM hard cap is breached (exit 3: a controlled stop beats a mid-step OOM when the desktop
+  spikes). ``model.safetensors`` is ``EncoderForMLM.state_dict_for_safetensors()`` (fp32 master
+  weights; the tied decoder alias is dropped and restored by ``load_tied_state_dict``).
+  ``trainer_state.pt`` carries optimizer, schedule, step/epoch/data cursor, every RNG state and
+  a config snapshot; ``--resume`` refuses a different model shape or a different plan.
+* **Logging.** Every ``log_every_steps`` optimizer steps: loss, lr, grad norm, tokens/s,
+  peak allocated/reserved VRAM for the interval (peak stats reset per interval), device memory
+  in use, elapsed and ETA, to the logger and to ``metrics.csv``. Val MLM loss and masked-token
+  accuracy every ``eval_every_steps`` on a fixed set of val windows with fixed masks.
+
+Hard rule honoured here: the TREC-06 holdout parquet is never opened by this script; the
+data pipeline's asserted-zero holdout-vs-pretrain overlap (data_report.md) is the tripwire.
+
+CLI::
+
+    python src/pretrain.py train [--run-name NAME] [--resume [PATH|latest]] [--epochs N]
+                                 [--max-steps N] [--max-hours H] [--micro-batch B]
+                                 [--grad-checkpointing] [--limit-docs N] [--checkpoint-every N]
+                                 [--log-every N] [--eval-every N] [--keep-checkpoints K] [--fresh]
+    python src/pretrain.py export --run-name NAME     # final/ from the latest checkpoint
+    python src/pretrain.py plot --run-name NAME       # reports/pretrain_curves.png
+"""
+
+from __future__ import annotations
+
+import os
+
+os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
+
+import argparse  # noqa: E402
+import csv  # noqa: E402
+import json  # noqa: E402
+import logging  # noqa: E402
+import math  # noqa: E402
+import random  # noqa: E402
+import re  # noqa: E402
+import shutil  # noqa: E402
+import signal  # noqa: E402
+import sys  # noqa: E402
+import time  # noqa: E402
+from dataclasses import asdict, dataclass  # noqa: E402
+from datetime import datetime  # noqa: E402
+from pathlib import Path  # noqa: E402
+from types import SimpleNamespace  # noqa: E402
+from typing import Any  # noqa: E402
+
+sys.path.insert(0, str(Path(__file__).parent))
+
+import numpy as np  # noqa: E402
+import pyarrow.parquet as pq  # noqa: E402
+import torch  # noqa: E402
+import torch.nn.functional as F  # noqa: E402
+from safetensors.torch import load_file, save_file  # noqa: E402
+from torch import Tensor  # noqa: E402
+
+import common  # noqa: E402
+from model import EncoderConfig, EncoderForMLM, count_parameters  # noqa: E402
+from smoke_test import build_optimizer, mlm_mask  # noqa: E402  (the measured code path)
+from tokenizer import load_tokenizer  # noqa: E402
+
+GIB = float(2**30)
+LOG = logging.getLogger("pretrain")
+VAL_FRACTION = 0.01                 # last 1% of pretraining docs -> val MLM loss only
+VAL_WINDOWS = 64                    # fixed windows (and fixed masks) scored at every eval
+VAL_MASK_SEED_OFFSET = 7919         # eval masks never share a generator with training
+ENCODE_CHUNK_DOCS = 4096
+WARMUP_FRACTION = 0.06              # config comment: warmup = min(1000, 6% of total steps)
+RUN_LOG_MIN_STEPS = 100             # dev runs shorter than this never touch RUN_LOG.md
+STEP_DIR_RE = re.compile(r"^step-(\d{7})$")
+METRICS_COLUMNS = [
+    "step", "epoch", "epoch_frac", "loss", "loss_avg", "lr", "grad_norm", "tokens_per_s",
+    "max_alloc_gb", "max_reserved_gb", "device_used_gb", "elapsed_s", "train_seconds_total",
+    "val_mlm_loss", "val_mlm_acc",
+]
+EXIT_OK, EXIT_CONFIG, EXIT_VRAM, EXIT_NAN = 0, 2, 3, 4
+
+
+# ================================================================================ data cache
+@dataclass
+class TokenCache:
+    train: np.ndarray          # 1-D uint16 token stream, [SEP] after every document
+    val: np.ndarray            # same, for the last 1% of documents
+    meta: dict[str, Any]
+
+
+def _cache_paths(processed: Path, limit_docs: int | None) -> tuple[Path, Path, Path]:
+    suffix = f".dev{limit_docs}" if limit_docs else ""
+    return (processed / f"pretrain_tokens{suffix}.npy",
+            processed / f"pretrain_val_tokens{suffix}.npy",
+            processed / f"pretrain_tokens{suffix}.json")
+
+
+def _encode_ids(tok, texts: list[str]) -> list[list[int]]:
+    fast = getattr(tok, "encode_batch_fast", None)
+    encs = fast(texts, add_special_tokens=False) if fast else tok.encode_batch(texts, add_special_tokens=False)
+    return [e.ids for e in encs]
+
+
+def load_or_build_token_cache(cfg: dict, limit_docs: int | None) -> TokenCache:
+    """Reuse the cached token stream when its sidecar matches the tokenizer and the parquet,
+    else tokenise the corpus once (~106M tokens; progress and ETA are logged per chunk)."""
+    processed = Path(cfg["paths"]["data_processed"])
+    parquet = processed / "pretrain.parquet"
+    tok_path = Path(cfg["paths"]["tokenizer"])
+    if not parquet.exists() or not tok_path.exists():
+        raise FileNotFoundError(f"need {parquet} and {tok_path}; run data_prep.py and tokenizer.py first")
+    rows = pq.ParquetFile(parquet).metadata.num_rows
+    n_docs = min(rows, limit_docs) if limit_docs else rows
+    tok_sha1 = common.sha1_hex(tok_path.read_text(encoding="utf-8"))
+    train_p, val_p, meta_p = _cache_paths(processed, limit_docs)
+
+    reason = None
+    if not (train_p.exists() and val_p.exists() and meta_p.exists()):
+        reason = "cache files missing"
+    else:
+        meta = common.read_json(meta_p)
+        if meta.get("tokenizer_sha1") != tok_sha1:
+            reason = f"tokenizer sha1 changed ({meta.get('tokenizer_sha1', '?')[:12]} -> {tok_sha1[:12]})"
+        elif meta.get("parquet_rows") != rows:
+            reason = f"parquet row count changed ({meta.get('parquet_rows')} -> {rows})"
+        elif meta.get("n_docs") != n_docs:
+            reason = f"document count changed ({meta.get('n_docs')} -> {n_docs})"
+    if reason is None:
+        meta = common.read_json(meta_p)
+        train = np.load(train_p)
+        val = np.load(val_p)
+        LOG.info("token cache reused: %s (%s tokens train / %s val, %d docs, tokenizer %s)",
+                 train_p.name, f"{len(train):,}", f"{len(val):,}", meta["n_docs"], tok_sha1[:12])
+        return TokenCache(train, val, meta)
+
+    LOG.info("rebuilding token cache: %s", reason)
+    tok = load_tokenizer(tok_path)
+    sep_id, mask_id = tok.token_to_id("[SEP]"), tok.token_to_id("[MASK]")
+    if sep_id is None or mask_id is None:
+        raise RuntimeError("tokenizer lacks [SEP]/[MASK]")
+    if tok.get_vocab_size() > np.iinfo(np.uint16).max + 1:
+        raise RuntimeError("vocab exceeds uint16; widen the cache dtype")
+    n_val_docs = max(1, int(n_docs * VAL_FRACTION))
+    split_at = n_docs - n_val_docs
+    LOG.info("tokenising %s docs from %s in chunks of %d (val = last %d docs); ~106M tokens expected for the full corpus",
+             f"{n_docs:,}", parquet.name, ENCODE_CHUNK_DOCS, n_val_docs)
+
+    train_chunks: list[np.ndarray] = []
+    val_chunks: list[np.ndarray] = []
+    done = chars = 0
+    n_tok_train = n_tok_val = 0
+    t0 = common.timer()
+    for batch in pq.ParquetFile(parquet).iter_batches(batch_size=ENCODE_CHUNK_DOCS, columns=["text"]):
+        if done >= n_docs:
+            break
+        texts = [t or "" for t in batch.column(0).to_pylist()][: n_docs - done]
+        chars += sum(map(len, texts))
+        for j, ids in enumerate(_encode_ids(tok, texts)):
+            arr = np.asarray(ids + [sep_id], dtype=np.uint16)
+            if done + j >= split_at:
+                val_chunks.append(arr)
+                n_tok_val += arr.size
+            else:
+                train_chunks.append(arr)
+                n_tok_train += arr.size
+        done += len(texts)
+        elapsed = common.timer() - t0
+        eta = elapsed / done * (n_docs - done) if done else 0.0
+        LOG.info("tokenising: %s/%s docs (%.0f%%), %.1fM tokens, %.0fs elapsed, ETA %s",
+                 f"{done:,}", f"{n_docs:,}", 100.0 * done / n_docs, (n_tok_train + n_tok_val) / 1e6, elapsed, fmt_dur(eta))
+    train = np.concatenate(train_chunks) if train_chunks else np.zeros(0, np.uint16)
+    val = np.concatenate(val_chunks) if val_chunks else np.zeros(0, np.uint16)
+    encode_s = common.timer() - t0
+    meta = {
+        "created": datetime.now().isoformat(timespec="seconds"),
+        "parquet": str(parquet), "parquet_rows": rows, "n_docs": done, "limit_docs": limit_docs,
+        "n_docs_train": min(done, split_at), "n_docs_val": max(0, done - split_at),
+        "val_rule": f"last {VAL_FRACTION:.0%} of documents by parquet order",
+        "tokenizer": str(tok_path), "tokenizer_sha1": tok_sha1, "vocab_size": tok.get_vocab_size(),
+        "sep_id": sep_id, "mask_id": mask_id, "dtype": "uint16",
+        "n_tokens_train": int(train.size), "n_tokens_val": int(val.size), "chars": chars,
+        "tokens_per_char": round((train.size + val.size) / max(1, chars), 5),
+        "encode_seconds": round(encode_s, 1),
+        "packing": "documents concatenated with [SEP] after each; windows of max_seq_len cut from the stream",
+    }
+    np.save(train_p, train)
+    np.save(val_p, val)
+    common.write_json(meta, meta_p)
+    LOG.info("token cache written: %s tokens train, %s val (%.4f tok/char) in %.0fs -> %s",
+             f"{train.size:,}", f"{val.size:,}", meta["tokens_per_char"], encode_s, train_p)
+    return TokenCache(train, val, meta)
+
+
+class WindowStore:
+    """Contiguous windows over a token stream; a batch is gathered on the CPU
+    (16 x 512 x 2 bytes) and moved to the device, which costs microseconds against a
+    ~350 ms micro-step and keeps the 212 MB corpus out of VRAM.
+
+    With ``prefix_id`` set (the [CLS] id) every window is ``[CLS]`` + ``seq_len - 1`` stream
+    tokens, as RoBERTa prepends ``<s>`` to each packed sequence. MLM never targets [CLS]
+    (specials are excluded from masking), but attention to and from position 0 is trained,
+    so the classifier's pooled [CLS] state does not start fine-tuning from an untouched row.
+    """
+
+    def __init__(self, stream: np.ndarray, seq_len: int, prefix_id: int | None = None) -> None:
+        self.prefix_id = prefix_id
+        self.body_len = seq_len - 1 if prefix_id is not None else seq_len
+        self.n = int(stream.size // self.body_len)
+        self.windows = stream[: self.n * self.body_len].reshape(self.n, self.body_len)
+
+    def batch(self, idx: np.ndarray, device: torch.device) -> Tensor:
+        w = torch.from_numpy(self.windows[idx].astype(np.int64))
+        if self.prefix_id is not None:
+            w = torch.cat([torch.full((w.shape[0], 1), self.prefix_id, dtype=torch.int64), w], dim=1)
+        return w.to(device, non_blocking=True)
+
+
+def epoch_permutation(seed: int, epoch: int, n: int) -> np.ndarray:
+    return np.random.default_rng(seed + epoch).permutation(n)
+
+
+# ================================================================================= schedule
+@dataclass
+class Schedule:
+    kind: str                  # "cosine" | "linear"
+    peak_lr: float
+    warmup_steps: int
+    total_steps: int
+    floor_ratio: float = 0.1   # decay target as a fraction of peak
+
+    @classmethod
+    def from_config(cls, pcfg: dict, total_steps: int) -> "Schedule":
+        name = str(pcfg.get("schedule", "cosine_to_10pct")).lower()
+        kind = "linear" if name.startswith("linear") else "cosine"
+        m = re.search(r"to_(\d+)pct", name)
+        floor = int(m.group(1)) / 100.0 if m else 0.1
+        frac = float(pcfg.get("warmup_fraction", WARMUP_FRACTION))
+        warmup = min(int(pcfg["warmup_steps"]), max(1, round(frac * total_steps)))
+        return cls(kind, float(pcfg["lr"]), warmup, int(total_steps), floor)
+
+    def lr_at(self, step: int) -> float:
+        """LR for optimizer step ``step`` (1-based). Past ``total_steps`` the floor holds."""
+        if step <= self.warmup_steps:
+            return self.peak_lr * step / self.warmup_steps
+        progress = min(1.0, (step - self.warmup_steps) / max(1, self.total_steps - self.warmup_steps))
+        decay = 0.5 * (1.0 + math.cos(math.pi * progress)) if self.kind == "cosine" else 1.0 - progress
+        return self.peak_lr * (self.floor_ratio + (1.0 - self.floor_ratio) * decay)
+
+
+# ============================================================================ run bookkeeping
+@dataclass
+class TrainState:
+    step: int = 0                    # optimizer steps completed
+    epoch: int = 0                   # 0-based epoch currently being trained
+    position: int = 0                # sequences consumed in the current epoch
+    tokens_seen: int = 0
+    train_seconds_total: float = 0.0 # wall time inside the training loop, summed over resumes
+    peak_reserved_gb: float = 0.0
+    peak_allocated_gb: float = 0.0
+    last_loss: float | None = None
+    last_val_loss: float | None = None
+    last_val_acc: float | None = None
+    invocations: int = 0
+
+
+class StopSignal:
+    """SIGINT/SIGTERM set a flag; the loop saves at the next optimizer-step boundary."""
+
+    def __init__(self) -> None:
+        self.name: str | None = None
+
+    def install(self) -> None:
+        for s in (signal.SIGINT, signal.SIGTERM):
+            signal.signal(s, self._handle)
+
+    def _handle(self, signum: int, frame: Any) -> None:
+        self.name = signal.Signals(signum).name
+        sys.stderr.write(f"\n[pretrain] {self.name} received: saving a checkpoint at the next optimizer step, "
+                         "then exiting 0 (kill -9 aborts without saving)\n")
+
+
+def fmt_dur(seconds: float) -> str:
+    seconds = max(0.0, float(seconds))
+    h, rem = divmod(int(seconds), 3600)
+    m, s = divmod(rem, 60)
+    return f"{h}h {m:02d}m {s:02d}s" if h else f"{m}m {s:02d}s"
+
+
+def list_step_dirs(run_dir: Path) -> list[Path]:
+    if not run_dir.exists():
+        return []
+    dirs = [p for p in run_dir.iterdir() if p.is_dir() and STEP_DIR_RE.match(p.name)]
+    return sorted(dirs, key=lambda p: int(STEP_DIR_RE.match(p.name).group(1)))
+
+
+def latest_step_dir(run_dir: Path) -> Path | None:
+    dirs = list_step_dirs(run_dir)
+    return dirs[-1] if dirs else None
+
+
+def rng_states(mask_gen: torch.Generator) -> dict[str, Any]:
+    return {
+        "python": random.getstate(),
+        "numpy": np.random.get_state(),
+        "torch_cpu": torch.get_rng_state(),
+        "torch_cuda": torch.cuda.get_rng_state_all(),
+        "mask_gen": mask_gen.get_state(),
+    }
+
+
+def restore_rng(states: dict[str, Any], mask_gen: torch.Generator) -> None:
+    random.setstate(states["python"])
+    np.random.set_state(states["numpy"])
+    torch.set_rng_state(states["torch_cpu"])
+    torch.cuda.set_rng_state_all(states["torch_cuda"])
+    mask_gen.set_state(states["mask_gen"])
+
+
+def config_snapshot(mcfg: EncoderConfig, pcfg: dict, seed: int, micro_batch: int, accum: int,
+                    sched: Schedule, steps_per_epoch: int, epochs: int, cache: TokenCache,
+                    n_windows: int, n_val_windows: int) -> dict[str, Any]:
+    model = mcfg.to_dict()
+    grad_ckpt = model.pop("grad_checkpointing")
+    return {
+        "model": model,
+        "grad_checkpointing": grad_ckpt,
+        "pretrain": {k: pcfg[k] for k in ("lr", "betas", "eps", "weight_decay", "grad_clip",
+                                           "mlm_probability", "mask_replace_probs", "schedule",
+                                           "effective_batch_sequences")},
+        "seed": seed,
+        "micro_batch": micro_batch,
+        "grad_accumulation": accum,
+        "effective_batch": micro_batch * accum,
+        "schedule": asdict(sched),
+        "steps_per_epoch": steps_per_epoch,
+        "epochs": epochs,
+        "total_steps": sched.total_steps,
+        "data": {"n_windows": n_windows, "n_val_windows": n_val_windows,
+                 "tokenizer_sha1": cache.meta["tokenizer_sha1"], "n_docs": cache.meta["n_docs"],
+                 "limit_docs": cache.meta.get("limit_docs"),
+                 "window": f"[CLS] + {mcfg.max_seq_len - 1} stream tokens"},
+    }
+
+
+def check_resume_compatible(saved: dict[str, Any], current: dict[str, Any]) -> None:
+    """Refuse to resume into a different model shape or a different plan; micro-batch and
+    gradient checkpointing may change (the effective batch may not, so the data cursor and
+    the schedule stay valid)."""
+    if saved["model"] != current["model"]:
+        diff = {k: (saved["model"].get(k), current["model"].get(k))
+                for k in set(saved["model"]) | set(current["model"]) if saved["model"].get(k) != current["model"].get(k)}
+        raise RuntimeError(f"refusing to resume: model config differs (saved, current) = {diff}")
+    for key in ("pretrain", "seed", "effective_batch", "schedule", "steps_per_epoch", "epochs", "total_steps", "data"):
+        if saved.get(key) != current.get(key):
+            raise RuntimeError(f"refusing to resume: '{key}' differs from the checkpoint "
+                               f"(saved={saved.get(key)!r}, current={current.get(key)!r}); "
+                               "a resumed run must continue the same schedule and data order")
+    for key in ("micro_batch", "grad_accumulation", "grad_checkpointing"):
+        if saved.get(key) != current.get(key):
+            LOG.warning("resume: %s changed %r -> %r (allowed; effective batch unchanged)", key, saved.get(key), current.get(key))
+
+
+# ============================================================================== checkpoints
+def save_checkpoint(run_dir: Path, model: EncoderForMLM, opt: torch.optim.Optimizer, sched: Schedule,
+                    state: TrainState, snapshot: dict[str, Any], mask_gen: torch.Generator,
+                    metrics_tail: list[dict[str, Any]], keep: int, reason: str) -> Path:
+    """Atomic: everything lands in ``step-NNNNNNN.tmp`` and is renamed at the end, so a kill
+    during the 4 GB write never leaves a half-written directory that ``--resume latest`` would
+    pick up. Older step dirs beyond ``keep`` are pruned afterwards."""
+    t0 = common.timer()
+    final = run_dir / f"step-{state.step:07d}"
+    tmp = run_dir / f"{final.name}.tmp"
+    if tmp.exists():
+        shutil.rmtree(tmp)
+    tmp.mkdir(parents=True)
+    sd = {k: v.detach().to("cpu", copy=True).contiguous() for k, v in model.state_dict_for_safetensors().items()}
+    save_file(sd, str(tmp / "model.safetensors"),
+              metadata={"format": "pt", "step": str(state.step), "config": json.dumps(model.config.to_dict())})
+    del sd
+    summary = {
+        "step": state.step, "epoch": state.epoch, "position": state.position, "tokens_seen": state.tokens_seen,
+        "train_seconds_total": round(state.train_seconds_total, 1),
+        "peak_reserved_gb": round(state.peak_reserved_gb, 3), "peak_allocated_gb": round(state.peak_allocated_gb, 3),
+        "last_loss": state.last_loss, "last_val_loss": state.last_val_loss, "last_val_acc": state.last_val_acc,
+        "lr": sched.lr_at(max(1, state.step)), "reason": reason, "saved_at": datetime.now().isoformat(timespec="seconds"),
+        "config": snapshot,
+    }
+    torch.save({
+        "optimizer": opt.state_dict(),
+        "scheduler": asdict(sched),
+        "step": state.step,
+        "epoch": state.epoch,
+        "micro_step_in_epoch": (state.step % max(1, snapshot["steps_per_epoch"])) * snapshot["grad_accumulation"],
+        "data_cursor": {"seed": snapshot["seed"], "epoch": state.epoch, "position": state.position,
+                        "n_windows": snapshot["data"]["n_windows"], "rule": "numpy default_rng(seed + epoch).permutation(n_windows)"},
+        "rng": rng_states(mask_gen),
+        "config": snapshot,
+        "state": asdict(state),
+        "metrics_tail": metrics_tail[-50:],
+    }, tmp / "trainer_state.pt")
+    common.write_json(summary, tmp / "summary.json")
+    if final.exists():
+        shutil.rmtree(final)
+    tmp.rename(final)
+    pruned = []
+    for old in list_step_dirs(run_dir)[:-keep] if keep > 0 else []:
+        shutil.rmtree(old)
+        pruned.append(old.name)
+    LOG.info("checkpoint saved: %s (%s) in %.1fs%s", final, reason, common.timer() - t0,
+             f"; pruned {pruned}" if pruned else "")
+    return final
+
+
+def load_checkpoint_into(step_dir: Path, model: EncoderForMLM, opt: torch.optim.Optimizer,
+                         mask_gen: torch.Generator, device: torch.device) -> tuple[dict[str, Any], TrainState]:
+    t0 = common.timer()
+    weights = load_file(str(step_dir / "model.safetensors"), device=str(device))
+    model.load_tied_state_dict(weights, strict=True)
+    del weights
+    saved = torch.load(step_dir / "trainer_state.pt", map_location="cpu", weights_only=False)
+    opt.load_state_dict(saved["optimizer"])
+    restore_rng(saved["rng"], mask_gen)
+    state = TrainState(**saved["state"])
+    LOG.info("resumed from %s: step %d, epoch %d, position %d sequences, %.1fs of training so far (load took %.1fs)",
+             step_dir, state.step, state.epoch, state.position, state.train_seconds_total, common.timer() - t0)
+    return saved, state
+
+
+# ================================================================================= metrics
+class MetricsCSV:
+    def __init__(self, path: Path) -> None:
+        self.path = path
+
+    def truncate_after(self, step: int) -> None:
+        """Drop rows past the resumed step so a run killed between checkpoints leaves no
+        duplicate or orphan steps in the record."""
+        if not self.path.exists():
+            return
+        with self.path.open("r", encoding="utf-8", newline="") as fh:
+            rows = list(csv.DictReader(fh))
+        keep = [r for r in rows if r.get("step", "").isdigit() and int(r["step"]) <= step]
+        if len(keep) != len(rows):
+            LOG.warning("metrics.csv: dropping %d rows past step %d from an earlier, unsaved stretch", len(rows) - len(keep), step)
+            self._write_all(keep)
+
+    def _write_all(self, rows: list[dict[str, Any]]) -> None:
+        with self.path.open("w", encoding="utf-8", newline="") as fh:
+            w = csv.DictWriter(fh, fieldnames=METRICS_COLUMNS, extrasaction="ignore")
+            w.writeheader()
+            w.writerows(rows)
+
+    def append(self, row: dict[str, Any]) -> None:
+        new = not self.path.exists() or self.path.stat().st_size == 0
+        with self.path.open("a", encoding="utf-8", newline="") as fh:
+            w = csv.DictWriter(fh, fieldnames=METRICS_COLUMNS, extrasaction="ignore")
+            if new:
+                w.writeheader()
+            w.writerow({k: ("" if row.get(k) is None else row[k]) for k in METRICS_COLUMNS})
+
+    def read(self) -> list[dict[str, str]]:
+        if not self.path.exists():
+            return []
+        with self.path.open("r", encoding="utf-8", newline="") as fh:
+            return list(csv.DictReader(fh))
+
+
+# ================================================================================== eval
+@torch.no_grad()
+def evaluate_mlm(model: EncoderForMLM, val_store: WindowStore, val_idx: np.ndarray, micro_batch: int,
+                 device: torch.device, mask_seed: int, pool_like: Any, vocab: int, p_mask: float,
+                 replace_probs: list[float]) -> tuple[float, float]:
+    """Mean MLM loss and masked-token accuracy on fixed val windows with fixed masks (the
+    generator is re-seeded every call, so successive evals score identical inputs)."""
+    was_training = model.training
+    model.eval()
+    gen = torch.Generator(device=device).manual_seed(mask_seed)
+    loss_sum = torch.zeros((), device=device, dtype=torch.float64)
+    correct = torch.zeros((), device=device, dtype=torch.int64)
+    n = 0
+    for i in range(0, len(val_idx), micro_batch):
+        ids = val_store.batch(val_idx[i:i + micro_batch], device)
+        inputs, labels, pos = mlm_mask(ids, pool_like, vocab, p_mask, replace_probs, gen)
+        with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+            logits = model(inputs, attention_mask=None, masked_positions=pos)
+        loss_sum += F.cross_entropy(logits.float(), labels, reduction="sum").double()
+        correct += (logits.argmax(dim=-1) == labels).sum()
+        n += int(labels.numel())
+    if was_training:
+        model.train()
+    return float(loss_sum) / max(1, n), float(correct) / max(1, n)
+
+
+# ============================================================================== final / log
+def write_final(run_dir: Path, step_dir: Path, cfg: dict, status: str) -> Path:
+    """``final/`` per the artifact contract: model.safetensors (copied from the chosen step
+    dir), config.json with the model config + tokenizer pointer + training summary, and a
+    copy of tokenizer.json so the directory is self-contained for finetune.py."""
+    final = run_dir / "final"
+    final.mkdir(parents=True, exist_ok=True)
+    shutil.copy2(step_dir / "model.safetensors", final / "model.safetensors")
+    tok_path = Path(cfg["paths"]["tokenizer"])
+    if tok_path.exists():
+        shutil.copy2(tok_path, final / "tokenizer.json")
+    summary_p = step_dir / "summary.json"
+    if summary_p.exists():
+        s = common.read_json(summary_p)
+    else:
+        ts = torch.load(step_dir / "trainer_state.pt", map_location="cpu", weights_only=False)
+        s = {**ts["state"], "config": ts["config"], "lr": None, "reason": "trainer_state.pt"}
+    snap = s["config"]
+    run_summary_p = run_dir / "run_summary.json"
+    run_summary = common.read_json(run_summary_p) if run_summary_p.exists() else {}
+    training = {
+        "run_name": run_dir.name, "status": status, "source_checkpoint": step_dir.name,
+        "steps": s["step"], "total_steps_planned": snap["total_steps"], "steps_per_epoch": snap["steps_per_epoch"],
+        "epochs_planned": snap["epochs"], "epochs_completed": round(s["step"] / max(1, snap["steps_per_epoch"]), 3),
+        "tokens_seen": s["tokens_seen"], "total_training_seconds": s["train_seconds_total"],
+        "total_training_time": fmt_dur(s["train_seconds_total"]),
+        "final_loss": s.get("last_loss"), "val_loss": s.get("last_val_loss"), "val_masked_acc": s.get("last_val_acc"),
+        "peak_vram_reserved_gb": s.get("peak_reserved_gb"), "peak_vram_allocated_gb": s.get("peak_allocated_gb"),
+        "micro_batch": snap["micro_batch"], "grad_accumulation": snap["grad_accumulation"], "effective_batch": snap["effective_batch"],
+        "precision": "bf16_autocast", "optimizer": "AdamW(fused) " + json.dumps(snap["pretrain"]),
+        "schedule": snap["schedule"], "mean_tokens_per_s": run_summary.get("mean_tokens_per_s"),
+        "data": snap["data"], "weights_dtype": "float32", "exported_at": datetime.now().isoformat(timespec="seconds"),
+    }
+    common.write_json({"model": EncoderConfig.from_dict({**snap["model"], "grad_checkpointing": False}).to_dict(),
+                       "tokenizer": "tokenizer.json", "training": training}, final / "config.json")
+    LOG.info("final/ written from %s -> %s", step_dir.name, final)
+    return final
+
+
+def next_run_log_number(path: Path) -> int:
+    nums = [int(m.group(1)) for m in re.finditer(r"^## (\d+)\.", path.read_text(encoding="utf-8"), re.M)] if path.exists() else []
+    return (max(nums) + 1) if nums else 0
+
+
+def run_log_block(run_dir: Path, snap: dict[str, Any], state: TrainState, status: str, mean_tok_s: float,
+                  logfile: str | None, counts: dict[str, Any]) -> str:
+    n = next_run_log_number(common.project_root() / "RUN_LOG.md")
+    sch = snap["schedule"]
+    lines = [
+        f"## {n}. {datetime.now():%Y-%m-%d %H:%M} — MLM pretraining run `{run_dir.name}` ({status})",
+        "",
+        f"**Run.** `python src/pretrain.py train --run-name {run_dir.name}`; {counts['total']:,} parameters (EncoderForMLM, tied head). "
+        f"micro-batch {snap['micro_batch']} x accumulation {snap['grad_accumulation']} = {snap['effective_batch']} sequences x "
+        f"{snap['model']['max_seq_len']} tokens per step; bf16 autocast, fused AdamW {snap['pretrain']['betas']} eps {snap['pretrain']['eps']} "
+        f"wd {snap['pretrain']['weight_decay']} (no decay on biases/LayerNorm), clip {snap['pretrain']['grad_clip']}; "
+        f"{sch['kind']} schedule peak {sch['peak_lr']:.1e} -> {sch['floor_ratio']:.0%} over {sch['total_steps']} planned steps "
+        f"({snap['epochs']} epochs x {snap['steps_per_epoch']} steps) with {sch['warmup_steps']} warmup steps; "
+        f"dynamic 15% masking 80/10/10, loss on masked positions only; {snap['data']['n_windows']:,} packed 512-token windows "
+        f"({snap['data']['n_docs']:,} docs), {snap['data']['n_val_windows']:,} val windows.",
+        "",
+        f"**Result.** {state.step} optimizer steps ({state.step / max(1, snap['steps_per_epoch']):.2f} epochs), "
+        f"{state.tokens_seen / 1e6:.1f}M tokens seen, total training time {fmt_dur(state.train_seconds_total)} "
+        f"({state.train_seconds_total:,.0f} s) over {state.invocations} invocation(s), mean {mean_tok_s:,.0f} tokens/s; "
+        f"peak VRAM reserved {state.peak_reserved_gb:.2f} GiB / allocated {state.peak_allocated_gb:.2f} GiB; "
+        f"final train MLM loss {state.last_loss if state.last_loss is None else f'{state.last_loss:.4f}'}, "
+        f"val MLM loss {state.last_val_loss if state.last_val_loss is None else f'{state.last_val_loss:.4f}'} "
+        f"(masked-token accuracy {state.last_val_acc if state.last_val_acc is None else f'{100 * state.last_val_acc:.1f}%'}).",
+        "",
+        f"**Artifacts.** {run_dir.relative_to(common.project_root()) if run_dir.is_relative_to(common.project_root()) else run_dir}/ "
+        f"(metrics.csv, run_summary.json, step-*/ and final/); log {logfile or 'logs/pretrain-*.log'}.",
+    ]
+    return "\n".join(lines)
+
+
+def current_logfile() -> str | None:
+    for h in LOG.handlers:
+        if isinstance(h, logging.FileHandler):
+            return h.baseFilename
+    return None
+
+
+def smoke_test_note(cfg: dict) -> str:
+    rep = Path(cfg["paths"]["reports"]) / "smoke_test.json"
+    try:
+        rows = {r["micro_batch"]: r for r in common.read_json(rep)["finder"]["no_ckpt"] if r.get("ok")}
+        parts = [f"{b}: {rows[b]['max_reserved_gib']:.2f} GiB reserved, {rows[b]['tokens_per_s']:,} tok/s" for b in sorted(rows)]
+        return f"smoke_test measured (no checkpointing) micro-batch {'; '.join(parts)} ({rep.name})"
+    except Exception:  # noqa: BLE001 - informational only
+        return "smoke_test measured 13.65 GiB reserved at micro-batch 16 and 17.65 GiB at 24 (reports/smoke_test.json)"
+
+
+# ================================================================================== train
+def resolve_run_and_resume(args: argparse.Namespace, cfg: dict) -> tuple[Path, Path | None]:
+    base = Path(cfg["paths"]["checkpoints"]) / "pretrain"
+    if args.resume is None:
+        return base / args.run_name, None
+    if args.resume == "latest":
+        run_dir = base / args.run_name
+        step_dir = latest_step_dir(run_dir)
+        if step_dir is None:
+            raise FileNotFoundError(f"--resume: no step-* checkpoint under {run_dir}")
+        return run_dir, step_dir
+    p = Path(args.resume).expanduser().resolve()
+    if (p / "trainer_state.pt").exists():
+        return p.parent, p
+    step_dir = latest_step_dir(p)
+    if step_dir is None:
+        raise FileNotFoundError(f"--resume {p}: neither a step dir nor a run dir with step-* checkpoints")
+    return p, step_dir
+
+
+def cmd_train(args: argparse.Namespace, cfg: dict) -> int:
+    t_invocation = common.timer()
+    global LOG
+    LOG = common.setup_logging("pretrain", cfg["paths"]["logs"])
+    device_info = common.print_device_info()
+    if not torch.cuda.is_available():
+        LOG.error("CUDA is required for pretraining")
+        return EXIT_CONFIG
+    LOG.info("PYTORCH_CUDA_ALLOC_CONF=%s | torch %s | %s", os.environ.get("PYTORCH_CUDA_ALLOC_CONF"), torch.__version__, device_info["device"])
+    seed = int(cfg["seed"])
+    common.set_seed(seed)
+    device = torch.device("cuda")
+    pcfg, vr = cfg["pretrain"], cfg["vram"]
+    hard_cap = float(vr["hard_cap_gb"])
+    mcfg = EncoderConfig.from_dict(cfg["model"])
+    if args.grad_checkpointing:
+        mcfg.grad_checkpointing = True
+    L = mcfg.max_seq_len
+
+    run_dir, resume_dir = resolve_run_and_resume(args, cfg)
+    if resume_dir is None and list_step_dirs(run_dir):
+        if args.fresh:
+            LOG.warning("--fresh: removing existing checkpoints and metrics in %s", run_dir)
+            shutil.rmtree(run_dir)
+        else:
+            LOG.error("%s already holds checkpoints %s; pass --resume to continue it, --fresh to start over, or a new --run-name",
+                      run_dir, [p.name for p in list_step_dirs(run_dir)])
+            return EXIT_CONFIG
+    run_dir.mkdir(parents=True, exist_ok=True)
+    saved_summary = common.read_json(resume_dir / "summary.json") if resume_dir and (resume_dir / "summary.json").exists() else None
+
+    # -------------------------------------------------------------- batch geometry
+    if args.micro_batch:
+        micro_batch = int(args.micro_batch)
+    elif saved_summary:
+        micro_batch = int(saved_summary["config"]["micro_batch"])
+    else:
+        micro_batch = int(pcfg["micro_batch_start"])
+    eff_target = int(pcfg["effective_batch_sequences"])
+    accum = max(1, round(eff_target / micro_batch))
+    eff_batch = accum * micro_batch
+    LOG.info("micro-batch %d x accumulation %d = effective %d sequences (%s tokens/step); %s",
+             micro_batch, accum, eff_batch, f"{eff_batch * L:,}", smoke_test_note(cfg))
+    if eff_batch != eff_target:
+        LOG.warning("realised effective batch %d != configured %d because %d is not a divisor of %d",
+                    eff_batch, eff_target, micro_batch, eff_target)
+
+    # ------------------------------------------------------------------------ data
+    # Refuse a mismatched --limit-docs BEFORE tokenising anything (resuming a dev run without
+    # --limit-docs would otherwise build the full 106M-token cache and then be refused).
+    if saved_summary is not None:
+        saved_limit = saved_summary["config"]["data"].get("limit_docs")
+        if args.limit_docs is None and saved_limit is not None:
+            LOG.warning("--resume: using the checkpoint's --limit-docs %s", saved_limit)
+            args.limit_docs = saved_limit
+        elif args.limit_docs != saved_limit:
+            LOG.error("--resume: --limit-docs %s differs from the checkpoint's %s; a resumed run must use the same data",
+                      args.limit_docs, saved_limit)
+            return EXIT_CONFIG
+    cache = load_or_build_token_cache(cfg, args.limit_docs)
+    cls_id = int(cfg["tokenizer"]["special_tokens"].index("[CLS]"))
+    train_store, val_store = WindowStore(cache.train, L, prefix_id=cls_id), WindowStore(cache.val, L, prefix_id=cls_id)
+    steps_per_epoch = train_store.n // eff_batch
+    if steps_per_epoch < 1:
+        LOG.error("only %d windows of %d tokens; need at least %d for one optimizer step", train_store.n, L, eff_batch)
+        return EXIT_CONFIG
+    epochs = int(args.epochs) if args.epochs else (int(saved_summary["config"]["epochs"]) if saved_summary else int(pcfg["epochs"]))
+    total_steps = epochs * steps_per_epoch
+    sched = Schedule.from_config(pcfg, total_steps)
+    LOG.info("data: %s train windows (%s tokens) -> %d steps/epoch x %d epochs = %d planned steps; %s val windows "
+             "(%d scored per eval); %d leftover windows per epoch are skipped",
+             f"{train_store.n:,}", f"{train_store.n * L:,}", steps_per_epoch, epochs, total_steps,
+             f"{val_store.n:,}", min(VAL_WINDOWS, val_store.n), train_store.n - steps_per_epoch * eff_batch)
+    LOG.info("schedule: %s warmup %d steps (= min(config %d, %.0f%% of %d)) -> peak %.2e -> floor %.0f%% at step %d",
+             sched.kind, sched.warmup_steps, int(pcfg["warmup_steps"]), 100 * float(pcfg.get("warmup_fraction", WARMUP_FRACTION)), total_steps,
+             sched.peak_lr, 100 * sched.floor_ratio, sched.total_steps)
+    val_idx = np.sort(np.random.default_rng(seed + VAL_MASK_SEED_OFFSET).choice(val_store.n, size=min(VAL_WINDOWS, val_store.n), replace=False)) if val_store.n else np.zeros(0, np.int64)
+
+    tok = load_tokenizer(cfg["paths"]["tokenizer"])
+    special_ids = [i for i in (tok.token_to_id(s) for s in cfg["tokenizer"]["special_tokens"]) if i is not None]
+    pool_like = SimpleNamespace(special_ids=special_ids, mask_id=tok.token_to_id("[MASK]"))
+    assert pool_like.mask_id == cache.meta["mask_id"], "tokenizer [MASK] id differs from the token cache"
+    p_mask, replace_probs = float(pcfg["mlm_probability"]), [float(x) for x in pcfg["mask_replace_probs"]]
+
+    # ----------------------------------------------------------------------- model
+    common.set_seed(seed)
+    with torch.device(device):
+        model = EncoderForMLM(mcfg)
+    model.train()
+    counts = count_parameters(model)
+    print(f"EncoderForMLM parameters: {counts['total']:,} total | embeddings {counts['embeddings']:,} | "
+          f"encoder layers {counts['encoder_layers']:,} | final norm {counts['final_norm']:,} | MLM head {counts['head']:,}"
+          f" | grad checkpointing {'ON' if mcfg.grad_checkpointing else 'off'}")
+    LOG.info("model: %s params %s", f"{counts['total']:,}", {k: v for k, v in mcfg.to_dict().items()})
+    opt = build_optimizer(model, sched.peak_lr, pcfg)
+    LOG.info("optimizer: %s fused=%s groups(decay=%d tensors, no_decay=%d tensors) betas=%s eps=%s wd=%s clip=%s",
+             type(opt).__name__, opt.defaults.get("fused"), len(opt.param_groups[0]["params"]), len(opt.param_groups[1]["params"]),
+             pcfg["betas"], pcfg["eps"], pcfg["weight_decay"], pcfg["grad_clip"])
+    mask_gen = torch.Generator(device=device).manual_seed(seed)
+    snapshot = config_snapshot(mcfg, pcfg, seed, micro_batch, accum, sched, steps_per_epoch, epochs, cache, train_store.n, val_store.n)
+    metrics = MetricsCSV(run_dir / "metrics.csv")
+
+    state = TrainState()
+    if resume_dir is not None:
+        saved, state = load_checkpoint_into(resume_dir, model, opt, mask_gen, device)
+        check_resume_compatible(saved["config"], snapshot)
+        metrics.truncate_after(state.step)
+    run_end = min(total_steps, int(args.max_steps)) if args.max_steps else total_steps
+    if state.step >= run_end:
+        LOG.warning("nothing to do: step %d already >= %d (planned %d, --max-steps %s); run_summary.json left untouched",
+                    state.step, run_end, total_steps, args.max_steps)
+        return EXIT_OK
+    state.invocations += 1
+    common.write_json({**asdict(state), "config": snapshot, "status": "running", "started": datetime.now().isoformat(timespec="seconds")},
+                      run_dir / "run_summary.json")
+    max_seconds = float(args.max_hours) * 3600.0 if args.max_hours else None
+    log_every = int(args.log_every or pcfg["log_every_steps"])
+    eval_every = int(pcfg["eval_every_steps"] if args.eval_every is None else args.eval_every)
+    ckpt_every = int(args.checkpoint_every or pcfg["checkpoint_every_steps"])
+    keep = int(args.keep_checkpoints)
+    LOG.info("run %s: steps %d -> %d of %d planned%s; log every %d, eval every %s, checkpoint every %d (keep %d), VRAM hard cap %.0f GiB",
+             run_dir.name, state.step, run_end, total_steps, f", --max-hours {args.max_hours}" if max_seconds else "",
+             log_every, eval_every or "never", ckpt_every, keep, hard_cap)
+
+    stop = StopSignal()
+    stop.install()
+    perm = epoch_permutation(seed, state.epoch, train_store.n)
+    LOG.info("data cursor: epoch %d, position %d, next window index %s (permutation seed %d)",
+             state.epoch, state.position, int(perm[state.position]) if state.position < train_store.n else "n/a", seed + state.epoch)
+
+    # ------------------------------------------------------------------- the loop
+    tokens_per_step = eff_batch * L
+    metrics_tail: list[dict[str, Any]] = []
+    last_saved_step = state.step if resume_dir is not None else -1
+    status, exit_code = "running", EXIT_OK
+    nonfinite_streak = 0
+    interval_losses: list[float] = []
+    interval_steps = 0
+    pause_s = 0.0                     # eval + checkpoint time, excluded from tokens/s
+    run_tokens = 0
+    run_train_s = 0.0
+
+    torch.cuda.synchronize()
+    torch.cuda.reset_peak_memory_stats()
+    t_loop0 = common.timer()          # starts AFTER checkpoint load: first tokens/s is clean
+    t_row = t_loop0
+    LOG.info("training loop started (setup took %.1fs)", t_loop0 - t_invocation)
+
+    def write_row(val: tuple[float, float] | None, lr: float, grad_norm: float, loss: float) -> None:
+        nonlocal t_row, interval_losses, interval_steps, pause_s, run_tokens, run_train_s
+        torch.cuda.synchronize()
+        now = common.timer()
+        wall = now - t_row
+        active = max(1e-6, wall - pause_s)
+        tok_s = interval_steps * tokens_per_step / active
+        run_tokens += interval_steps * tokens_per_step
+        run_train_s += active
+        vram = common.vram_stats()
+        free, total = torch.cuda.mem_get_info()
+        state.peak_reserved_gb = max(state.peak_reserved_gb, vram["max_reserved_gb"])
+        state.peak_allocated_gb = max(state.peak_allocated_gb, vram["max_allocated_gb"])
+        elapsed = now - t_loop0
+        state.train_seconds_total += wall
+        row = {
+            "step": state.step, "epoch": (state.step - 1) // steps_per_epoch + 1,
+            "epoch_frac": round(state.step / steps_per_epoch, 4),
+            "loss": round(loss, 5), "loss_avg": round(sum(interval_losses) / max(1, len(interval_losses)), 5),
+            "lr": f"{lr:.6e}", "grad_norm": round(grad_norm, 4), "tokens_per_s": round(tok_s),
+            "max_alloc_gb": round(vram["max_allocated_gb"], 3), "max_reserved_gb": round(vram["max_reserved_gb"], 3),
+            "device_used_gb": round((total - free) / GIB, 3), "elapsed_s": round(elapsed, 1),
+            "train_seconds_total": round(state.train_seconds_total, 1),
+            "val_mlm_loss": None if val is None else round(val[0], 5), "val_mlm_acc": None if val is None else round(val[1], 5),
+        }
+        metrics.append(row)
+        metrics_tail.append(row)
+        del metrics_tail[:-50]
+        remaining = run_end - state.step
+        eta = remaining * (active / max(1, interval_steps))
+        LOG.info("step %d/%d ep %d | loss %.4f (avg %.4f) | lr %.2e | gnorm %.2f | %s tok/s | alloc %.2f GiB res %.2f GiB used %.2f GiB"
+                 " | elapsed %s | ETA %s%s",
+                 state.step, run_end, row["epoch"], loss, row["loss_avg"], lr, grad_norm, f"{tok_s:,.0f}",
+                 vram["max_allocated_gb"], vram["max_reserved_gb"], row["device_used_gb"], fmt_dur(elapsed), fmt_dur(eta),
+                 "" if val is None else f" | val_mlm_loss {val[0]:.4f} val_acc {100 * val[1]:.2f}%")
+        t_row = now
+        interval_losses, interval_steps, pause_s = [], 0, 0.0
+        torch.cuda.reset_peak_memory_stats()
+
+    def checkpoint(reason: str) -> None:
+        nonlocal last_saved_step, pause_s
+        if last_saved_step == state.step:
+            return
+        t0 = common.timer()
+        save_checkpoint(run_dir, model, opt, sched, state, snapshot, mask_gen, metrics_tail, keep, reason)
+        last_saved_step = state.step
+        pause_s += common.timer() - t0
+
+    while state.step < run_end:
+        lr = sched.lr_at(state.step + 1)
+        for g in opt.param_groups:
+            g["lr"] = lr
+        micro_losses = []
+        phase = "forward/backward"
+        try:
+            for _ in range(accum):
+                idx = perm[state.position:state.position + micro_batch]
+                state.position += micro_batch
+                ids = train_store.batch(idx, device)
+                inputs, labels, pos = mlm_mask(ids, pool_like, mcfg.vocab_size, p_mask, replace_probs, mask_gen)
+                with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+                    logits = model(inputs, attention_mask=None, masked_positions=pos)
+                    loss = F.cross_entropy(logits, labels)
+                (loss / accum).backward()
+                micro_losses.append(float(loss.detach()))
+            step_loss = sum(micro_losses) / accum
+            if not math.isfinite(step_loss):
+                nonfinite_streak += 1
+                opt.zero_grad(set_to_none=True)
+                LOG.error("non-finite loss %r at step %d (streak %d); optimizer step skipped", step_loss, state.step + 1, nonfinite_streak)
+                if nonfinite_streak >= 5:
+                    status, exit_code = "diverged", EXIT_NAN
+                    break
+                state.step += 1
+            else:
+                nonfinite_streak = 0
+                phase = "optimizer step"
+                grad_norm = float(torch.nn.utils.clip_grad_norm_(model.parameters(), float(pcfg["grad_clip"])))
+                opt.step()
+                opt.zero_grad(set_to_none=True)
+                state.step += 1
+                state.last_loss = step_loss
+            state.tokens_seen += tokens_per_step
+            interval_losses.append(step_loss)
+            interval_steps += 1
+
+            epoch_end = state.step % steps_per_epoch == 0
+            if epoch_end:
+                LOG.info("epoch %d/%d complete at step %d", state.epoch + 1, epochs, state.step)
+                state.epoch += 1
+                state.position = 0
+                perm = epoch_permutation(seed, state.epoch, train_store.n)
+
+            val = None
+            finished = state.step >= run_end
+            if eval_every and (state.step % eval_every == 0 or finished) and len(val_idx):
+                phase = "eval"
+                t0 = common.timer()
+                val = evaluate_mlm(model, val_store, val_idx, micro_batch, device, seed + VAL_MASK_SEED_OFFSET, pool_like,
+                                   mcfg.vocab_size, p_mask, replace_probs)
+                state.last_val_loss, state.last_val_acc = val
+                pause_s += common.timer() - t0
+        except torch.OutOfMemoryError as e:
+            # Another process grabbing VRAM (desktop spike, a second job) OOMs this one even
+            # though its own reserved peak is under the cap. In the forward/backward phase the
+            # weights and optimizer state are intact, so the partial step is discarded; after a
+            # completed optimizer step (eval phase) the step counts. Either way the data cursor
+            # is rewound to the last completed step, a checkpoint is saved and the resume
+            # continues from exactly there. An OOM inside opt.step itself is the one case where
+            # weights may be partially updated; it is logged as such.
+            ids = inputs = labels = pos = logits = loss = None   # drop the half-built graph before saving
+            opt.zero_grad(set_to_none=True)
+            torch.cuda.empty_cache()
+            model.train()
+            state.position = (state.step % steps_per_epoch) * eff_batch
+            free, total = torch.cuda.mem_get_info()
+            LOG.error("CUDA OOM during the %s of step %d (%s); device has %.2f of %.2f GiB in use across all processes, this process "
+                      "reserved %.2f GiB. Saving a checkpoint at step %d and stopping (exit %d).%s", phase, state.step + (phase != "eval"),
+                      str(e).splitlines()[0][:160], (total - free) / GIB, total / GIB, torch.cuda.memory_reserved() / GIB, state.step,
+                      EXIT_VRAM, " WARNING: the OOM hit inside opt.step; weights may be partially updated." if phase == "optimizer step" else "")
+            status, exit_code = "oom", EXIT_VRAM
+            break
+        if state.step % log_every == 0 or val is not None or finished or stop.name or epoch_end:
+            write_row(val, lr, grad_norm if math.isfinite(step_loss) else float("nan"), step_loss)
+
+        reserved_peak = torch.cuda.max_memory_reserved() / GIB
+        if reserved_peak > hard_cap:
+            state.peak_reserved_gb = max(state.peak_reserved_gb, reserved_peak)
+            LOG.error("VRAM guard: reserved peak %.2f GiB exceeds the %.0f GiB hard cap; saving and stopping (exit %d). "
+                      "Resume with a smaller --micro-batch or --grad-checkpointing.", reserved_peak, hard_cap, EXIT_VRAM)
+            status, exit_code = "vram_guard", EXIT_VRAM
+            break
+        if finished:
+            status = "complete" if state.step >= total_steps else "max_steps"
+            break
+        if stop.name:
+            status = f"signal:{stop.name}"
+            break
+        if max_seconds is not None and (common.timer() - t_loop0) >= max_seconds:
+            LOG.warning("--max-hours %.2f reached at step %d; saving a checkpoint and exiting 0 (resume with --resume)", args.max_hours, state.step)
+            status = "max_hours"
+            break
+        if epoch_end:
+            checkpoint(f"epoch {state.epoch} end")
+        elif state.step % ckpt_every == 0:
+            checkpoint("periodic")
+
+    # -------------------------------------------------------------------- wrap up
+    if interval_steps:
+        write_row(None, lr, float("nan"), state.last_loss if state.last_loss is not None else float("nan"))
+    checkpoint(status)
+    run_wall = common.timer() - t_loop0
+    mean_tok_s = run_tokens / max(1e-6, run_train_s)
+    summary = {
+        **asdict(state), "status": status, "exit_code": exit_code, "run_end_step": run_end,
+        "this_invocation": {"steps": run_tokens // tokens_per_step, "tokens": run_tokens, "loop_seconds": round(run_wall, 1),
+                            "mean_tokens_per_s": round(mean_tok_s), "setup_seconds": round(t_loop0 - t_invocation, 1)},
+        "mean_tokens_per_s": round(mean_tok_s), "config": snapshot, "params": counts,
+        "finished": datetime.now().isoformat(timespec="seconds"), "log": current_logfile(),
+    }
+    common.write_json(summary, run_dir / "run_summary.json")
+    print(f"\nTotal training time (this invocation): {fmt_dur(run_wall)} ({run_wall:.1f} s) for {run_tokens // tokens_per_step} steps, "
+          f"{mean_tok_s:,.0f} tokens/s mean")
+    print(f"Total training time (all invocations): {fmt_dur(state.train_seconds_total)} ({state.train_seconds_total:.1f} s), "
+          f"{state.step} steps, {state.tokens_seen / 1e6:.1f}M tokens, peak VRAM reserved {state.peak_reserved_gb:.2f} GiB")
+    LOG.info("run %s stopped: %s at step %d/%d (exit %d)", run_dir.name, status, state.step, total_steps, exit_code)
+    if status == "complete":
+        write_final(run_dir, run_dir / f"step-{state.step:07d}", cfg, status)
+    if status in ("complete", "max_steps", "max_hours") and state.step >= RUN_LOG_MIN_STEPS:
+        common.append_run_log(run_log_block(run_dir, snapshot, state, status, mean_tok_s, current_logfile(), counts))
+        LOG.info("RUN_LOG.md updated")
+    else:
+        LOG.info("RUN_LOG.md not updated (status %s, step %d; needs a normal stop at >= %d steps)", status, state.step, RUN_LOG_MIN_STEPS)
+    return exit_code
+
+
+# ============================================================================ export / plot
+def cmd_export(args: argparse.Namespace, cfg: dict) -> int:
+    global LOG
+    LOG = common.setup_logging("pretrain", cfg["paths"]["logs"])
+    run_dir = Path(cfg["paths"]["checkpoints"]) / "pretrain" / args.run_name
+    step_dir = latest_step_dir(run_dir)
+    if step_dir is None:
+        LOG.error("no step-* checkpoint under %s", run_dir)
+        return EXIT_CONFIG
+    summary_p = run_dir / "run_summary.json"
+    status = common.read_json(summary_p).get("status", "exported") if summary_p.exists() else "exported"
+    final = write_final(run_dir, step_dir, cfg, status)
+    cfgj = common.read_json(final / "config.json")
+    print(json.dumps(cfgj["training"], indent=2))
+    print(f"final/: {sorted(p.name for p in final.iterdir())} -> {final}")
+    return EXIT_OK
+
+
+def cmd_plot(args: argparse.Namespace, cfg: dict) -> int:
+    global LOG
+    LOG = common.setup_logging("pretrain", cfg["paths"]["logs"])
+    import matplotlib
+    matplotlib.use("Agg")
+    import matplotlib.pyplot as plt
+
+    run_dir = Path(cfg["paths"]["checkpoints"]) / "pretrain" / args.run_name
+    rows = MetricsCSV(run_dir / "metrics.csv").read()
+    if not rows:
+        LOG.error("no metrics.csv under %s", run_dir)
+        return EXIT_CONFIG
+    steps = np.array([int(r["step"]) for r in rows])
+    loss = np.array([float(r["loss"]) for r in rows])
+    lr = np.array([float(r["lr"]) for r in rows])
+    val_rows = [r for r in rows if r.get("val_mlm_loss")]
+    fig, ax = plt.subplots(figsize=(9, 5))
+    ax.plot(steps, loss, color="tab:blue", lw=1.2, label="train MLM loss (per logged step)")
+    if val_rows:
+        ax.plot([int(r["step"]) for r in val_rows], [float(r["val_mlm_loss"]) for r in val_rows],
+                "o-", color="tab:orange", lw=1.5, ms=4, label="val MLM loss")
+    ax.axhline(math.log(int(cfg["model"]["vocab_size"])), color="grey", ls=":", lw=1, label=f"ln(V) = {math.log(int(cfg['model']['vocab_size'])):.2f}")
+    ax.set_xlabel("optimizer step")
+    ax.set_ylabel("MLM cross-entropy")
+    ax.set_title(f"Pretraining {args.run_name}: {len(rows)} logged steps, last train loss {loss[-1]:.3f}"
+                 + (f", last val loss {float(val_rows[-1]['val_mlm_loss']):.3f}" if val_rows else ""))
+    ax.grid(alpha=0.3)
+    ax2 = ax.twinx()
+    ax2.plot(steps, lr, color="tab:green", lw=1, alpha=0.7, label="learning rate")
+    ax2.set_ylabel("learning rate")
+    ax2.ticklabel_format(axis="y", style="sci", scilimits=(0, 0))
+    h1, l1 = ax.get_legend_handles_labels()
+    h2, l2 = ax2.get_legend_handles_labels()
+    ax.legend(h1 + h2, l1 + l2, loc="upper right", fontsize=9)
+    out = Path(cfg["paths"]["reports"]) / "pretrain_curves.png"
+    out.parent.mkdir(parents=True, exist_ok=True)
+    fig.tight_layout()
+    fig.savefig(out, dpi=150)
+    LOG.info("wrote %s (%d rows, %d val points)", out, len(rows), len(val_rows))
+    print(out)
+    return EXIT_OK
+
+
+# ==================================================================================== CLI
+def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--config", default="config.yaml")
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    tr = sub.add_parser("train", help="MLM pretraining (resumable)")
+    tr.add_argument("--run-name", default="large-24L")
+    tr.add_argument("--resume", nargs="?", const="latest", default=None, metavar="PATH",
+                    help="resume from a step dir / run dir, or 'latest' under the run (default when the flag has no value)")
+    tr.add_argument("--epochs", type=int, default=None, help="planned epochs (config.pretrain.epochs); fixes the schedule length")
+    tr.add_argument("--max-steps", type=int, default=None, help="stop this invocation once the GLOBAL step reaches N (schedule unchanged)")
+    tr.add_argument("--max-hours", type=float, default=None, help="stop this invocation after H hours of training: save, exit 0")
+    tr.add_argument("--micro-batch", type=int, default=None, help="default config.pretrain.micro_batch_start (16), or the checkpoint's on resume")
+    tr.add_argument("--grad-checkpointing", action="store_true", help="activation checkpointing (config.model.grad_checkpointing)")
+    tr.add_argument("--limit-docs", type=int, default=None, help="dev: tokenise only the first N documents (separate cache files)")
+    tr.add_argument("--checkpoint-every", type=int, default=None)
+    tr.add_argument("--log-every", type=int, default=None)
+    tr.add_argument("--eval-every", type=int, default=None, help="0 disables periodic eval")
+    tr.add_argument("--keep-checkpoints", type=int, default=3)
+    tr.add_argument("--fresh", action="store_true", help="start over in a run dir that already holds checkpoints (deletes them)")
+    ex = sub.add_parser("export", help="write final/ (model.safetensors + config.json + tokenizer.json) from the latest checkpoint")
+    ex.add_argument("--run-name", required=True)
+    pl = sub.add_parser("plot", help="reports/pretrain_curves.png from metrics.csv")
+    pl.add_argument("--run-name", required=True)
+    return ap.parse_args(argv)
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = parse_args(argv)
+    cfg = common.load_config(args.config)
+    if args.cmd == "train":
+        return cmd_train(args, cfg)
+    if args.cmd == "export":
+        return cmd_export(args, cfg)
+    return cmd_plot(args, cfg)
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/presentation/src/smoke_test.py b/presentation/src/smoke_test.py
new file mode 100644
index 0000000000000000000000000000000000000000..900527e4a8c02eab087b819dfe9686d34a46be03
--- /dev/null
+++ b/presentation/src/smoke_test.py
@@ -0,0 +1,795 @@
+"""Smoke test: prove the training pipeline and measure VRAM / throughput before any real run.
+
+Sequence (every step logs; results land in reports/smoke_test.json and a markdown summary
+is printed at the end):
+
+1. device info, seed, config; ``PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True`` set before
+   CUDA initialises (variable-size probing fragments the caching allocator otherwise).
+2. build ``EncoderForMLM`` from config.yaml and print ``count_parameters``.
+3. data: packed real pretraining text if data/processed/{pretrain.parquet,tokenizer.json}
+   exist, else random token ids. MLM masking exactly as configured (15%, 80/10/10, loss on
+   masked positions only, specials never masked).
+4. micro-batch finder: per candidate, 1 warmup + 3 timed optimizer steps, each made of
+   two accumulated fwd/bwd micro-steps under bf16 autocast + fused AdamW. Two, not one,
+   because production accumulates: from the second micro-step on the fp32 gradients
+   (1.26 GiB) are resident while the next activation set is built, and that is the real
+   high-water mark (a single fwd/bwd/opt step under-reports it by one gradient copy).
+   Peak allocated/reserved recorded, OOMs caught; once without and once with gradient
+   checkpointing. The choice follows config.vram: the largest candidate whose reserved
+   peak sits in [target_gb_low, target_gb_high], else the largest under target_gb_high,
+   never above hard_cap_gb. The throughput run's own reserved peak is then checked
+   against the same cap and band as a second opinion.
+5. throughput at the chosen micro-batch (no checkpointing): CUDA-event-timed micro-steps
+   with the real gradient-accumulation pattern (accumulation = round(256 / micro-batch);
+   the effective batch actually realised is reported, and a warning is logged when it is
+   not exactly 256), an isolated optimizer-step cost, tokens/s, and projected hours per
+   epoch / per 3 epochs. Epoch tokens come from --epoch-tokens, else from the data
+   manifest's pretraining character count times the tokens/char measured on the packed
+   sample, else the 117M estimate. Plus a check of which SDPA backend actually runs on
+   the mask-free (Flash) and masked (mem-efficient) paths.
+6. mini convergence check from a fresh init: 30 steps at LR 1e-4 with a 10-step warmup.
+   The first loss should sit near ln(V) = 10.40; on real text it should fall visibly.
+7. inference latency of ``EncoderForClassification`` for one 512-token email: GPU bf16
+   (100 samples), CPU fp32 at 512 and 128 tokens (10 samples), versus the instructor's
+   0.074 s/sample benchmark.
+8. write the JSON report and print the summary.
+
+CLI: python src/smoke_test.py [--steps 30] [--skip-cpu] [--candidates 16,24,12,8,4]
+"""
+
+import os
+
+os.environ.setdefault("PYTORCH_CUDA_ALLOC_CONF", "expandable_segments:True")
+
+import argparse  # noqa: E402
+import gc  # noqa: E402
+import logging  # noqa: E402
+import math  # noqa: E402
+import sys  # noqa: E402
+import time  # noqa: E402
+import warnings  # noqa: E402
+from dataclasses import asdict, dataclass  # noqa: E402
+from pathlib import Path  # noqa: E402
+from typing import Any  # noqa: E402
+
+sys.path.insert(0, str(Path(__file__).parent))
+
+import torch  # noqa: E402
+import torch.nn.functional as F  # noqa: E402
+from torch import Tensor  # noqa: E402
+from torch.nn.attention import SDPBackend, sdpa_kernel  # noqa: E402
+
+import common  # noqa: E402
+from model import EncoderConfig, EncoderForClassification, EncoderForMLM, count_parameters  # noqa: E402
+
+GIB = float(2**30)
+INSTRUCTOR_BENCHMARK_S = 0.074
+DEFAULT_EPOCH_TOKENS = 117e6          # verify_data.md: ~117M tokens/epoch at 4 chars/token
+PROBE_ACCUM = 2                       # micro-steps per finder step: grads resident during the 2nd
+VRAM_UNIT_NOTE = "config vram.*_gb values are read as GiB and compared with torch.cuda.max_memory_reserved() / 2**30"
+LOG = logging.getLogger("smoke_test")
+
+
+# ------------------------------------------------------------------------------ data
+@dataclass
+class TokenPool:
+    """Fixed-length token sequences the test samples batches from."""
+
+    seqs: Tensor                 # [n, L] int64 on the training device
+    special_ids: list[int]       # never masked, never used as MLM targets
+    mask_id: int
+    source: str                  # "real" | "random"
+    n_docs: int
+    detail: str
+    tokens_per_char: float | None = None   # measured on the packed sample (real data only)
+
+    @property
+    def n(self) -> int:
+        return int(self.seqs.shape[0])
+
+    def batch(self, size: int, gen: torch.Generator) -> Tensor:
+        idx = torch.randint(self.n, (size,), device=self.seqs.device, generator=gen)
+        return self.seqs[idx]
+
+
+def _special_ids_from_config(cfg: dict) -> tuple[list[int], int]:
+    specials = cfg["tokenizer"]["special_tokens"]
+    return list(range(len(specials))), specials.index("[MASK]")
+
+
+def _text_columns(names: set[str], path: Path) -> list[str]:
+    cols = ["text"] if "text" in names else [c for c in ("subject", "body") if c in names]
+    if not cols:
+        raise ValueError(f"{path}: no text column among {sorted(names)}")
+    return cols
+
+
+def _texts(batch) -> list[str]:
+    """Model input text per row: the ``text`` column, else subject + blank line + body."""
+    names = set(batch.schema.names)
+    if "text" in names:
+        return [t or "" for t in batch.column("text").to_pylist()]
+    subj = batch.column("subject").to_pylist() if "subject" in names else [""] * batch.num_rows
+    body = batch.column("body").to_pylist()
+    return [f"{s or ''}\n\n{b or ''}" for s, b in zip(subj, body)]
+
+
+def _assert_no_holdout_overlap(texts: list[str], holdout: Path) -> None:
+    """The hard no-leak rule, checked on the exact documents this run packs: none may
+    equal a held-out TREC-06 email under the verify_data.md sec. 7 normalisation
+    (lowercase, [a-z0-9] only). Cheap (one sha1 per row) and independent of whatever
+    bookkeeping the data pipeline recorded."""
+    if not holdout.exists():
+        LOG.warning("no holdout parquet at %s; in-code leak check skipped", holdout)
+        return
+    import pyarrow.parquet as pq
+
+    pf = pq.ParquetFile(holdout)
+    cols = _text_columns(set(pf.schema_arrow.names), holdout)
+    hold: set[str] = set()
+    for batch in pf.iter_batches(batch_size=4096, columns=cols):
+        hold.update(common.sha1_hex(n) for n in map(common.normalize_text, _texts(batch)) if n)
+    hits = sum(common.sha1_hex(n) in hold for n in map(common.normalize_text, texts) if n)
+    assert hits == 0, f"{hits} pretraining documents match held-out emails in {holdout}"
+    LOG.info("no-leak check: 0 of %d packed docs match the %d holdout emails", len(texts), len(hold))
+
+
+def build_token_pool(cfg: dict, mcfg: EncoderConfig, device: torch.device,
+                     n_docs: int, random_seqs: int) -> TokenPool:
+    """Real packed text when the data agent's artefacts exist, else random ids.
+
+    Packing follows the pretraining recipe: documents are concatenated with [SEP]
+    between them and cut into max_seq_len windows (RoBERTa FULL-SENTENCES), which is
+    what keeps the Flash backend eligible (no padding -> no mask).
+    """
+    L = mcfg.max_seq_len
+    processed = Path(cfg["paths"]["data_processed"])
+    parquet = processed / "pretrain.parquet"
+    tok_path = Path(cfg["paths"]["tokenizer"])
+    if parquet.exists() and tok_path.exists():
+        import pyarrow.parquet as pq
+        from tokenizers import Tokenizer
+
+        pf = pq.ParquetFile(parquet)
+        names = set(pf.schema_arrow.names)
+        source_col = next((c for c in ("source", "dataset_name", "dataset") if c in names), None)
+        cols = _text_columns(names, parquet) + ([source_col] if source_col else [])
+        table = next(pf.iter_batches(batch_size=n_docs, columns=cols))
+        if source_col:
+            # Hard rule: the TREC-06 holdout must never reach pretraining text.
+            srcs = {str(s).lower().replace("_", "-") for s in table.column(source_col).to_pylist()}
+            assert not any("trec-06" in s or "trec06" in s for s in srcs), f"TREC-06 rows in {parquet}"
+        texts = _texts(table)
+        _assert_no_holdout_overlap(texts, processed / "test_holdout.parquet")
+
+        tok = Tokenizer.from_file(str(tok_path))
+        special_ids = [i for i in (tok.token_to_id(s) for s in cfg["tokenizer"]["special_tokens"]) if i is not None]
+        sep_id, mask_id = tok.token_to_id("[SEP]"), tok.token_to_id("[MASK]")
+        assert sep_id is not None and mask_id is not None, "tokenizer lacks [SEP]/[MASK]"
+        assert tok.get_vocab_size() <= mcfg.vocab_size, "tokenizer vocab larger than model vocab"
+        stream: list[int] = []
+        for enc in tok.encode_batch(texts, add_special_tokens=False):
+            stream.extend(enc.ids)
+            stream.append(sep_id)
+        n_seq = len(stream) // L
+        if n_seq < 1:
+            raise ValueError("not enough text to pack a single sequence")
+        seqs = torch.tensor(stream[: n_seq * L], dtype=torch.long).view(n_seq, L).to(device)
+        n_chars = sum(len(t) for t in texts)
+        tpc = len(stream) / n_chars if n_chars else None
+        return TokenPool(seqs, special_ids, mask_id, "real", len(texts),
+                         f"{len(texts)} docs ({n_chars:,} chars) -> {len(stream):,} tokens -> {n_seq} packed x {L}",
+                         tokens_per_char=tpc)
+
+    special_ids, mask_id = _special_ids_from_config(cfg)
+    gen = torch.Generator(device="cpu").manual_seed(cfg["seed"])
+    seqs = torch.randint(len(special_ids), mcfg.vocab_size, (random_seqs, L), generator=gen).to(device)
+    return TokenPool(seqs, special_ids, mask_id, "random", 0,
+                     f"{random_seqs} random x {L} ids in [{len(special_ids)}, {mcfg.vocab_size})")
+
+
+def mlm_mask(ids: Tensor, pool: TokenPool, vocab_size: int, p: float, replace_probs: list[float],
+             gen: torch.Generator) -> tuple[Tensor, Tensor, Tensor]:
+    """BERT dynamic masking: select ``p`` of the non-special positions; of those, replace
+    80% with [MASK], 10% with a random token, leave 10% unchanged. Returns
+    ``(inputs, labels_at_masked, masked_positions)`` so the model can gather before the head.
+    """
+    special = torch.isin(ids, torch.tensor(pool.special_ids, device=ids.device))
+    sel = (torch.rand(ids.shape, device=ids.device, generator=gen) < p) & ~special
+    if not bool(sel.any()):
+        first = (~special).view(-1).nonzero()[0]
+        sel.view(-1)[first] = True
+    labels = ids[sel]
+    inputs = ids.clone()
+    r = torch.rand(ids.shape, device=ids.device, generator=gen)
+    p_mask, p_rand, _ = replace_probs
+    to_mask = sel & (r < p_mask)
+    to_rand = sel & (r >= p_mask) & (r < p_mask + p_rand)
+    inputs[to_mask] = pool.mask_id
+    n_rand = int(to_rand.sum())
+    if n_rand:
+        inputs[to_rand] = torch.randint(len(pool.special_ids), vocab_size, (n_rand,),
+                                        device=ids.device, generator=gen)
+    return inputs, labels, sel
+
+
+# ------------------------------------------------------------------------ training bits
+def build_model(mcfg: EncoderConfig, device: torch.device, seed: int) -> EncoderForMLM:
+    common.set_seed(seed)
+    with torch.device(device):
+        model = EncoderForMLM(mcfg)
+    return model.train()
+
+
+def build_optimizer(model: torch.nn.Module, lr: float, pcfg: dict) -> torch.optim.AdamW:
+    """AdamW with weight decay only on matrices (biases and LayerNorm excluded), fused."""
+    decay = [p for p in model.parameters() if p.requires_grad and p.ndim >= 2]
+    no_decay = [p for p in model.parameters() if p.requires_grad and p.ndim < 2]
+    groups = [{"params": decay, "weight_decay": pcfg["weight_decay"]},
+              {"params": no_decay, "weight_decay": 0.0}]
+    kw: dict[str, Any] = dict(lr=lr, betas=tuple(pcfg["betas"]), eps=pcfg["eps"])
+    try:
+        return torch.optim.AdamW(groups, fused=True, **kw)
+    except (TypeError, RuntimeError) as e:  # fused unavailable on this build/device
+        LOG.warning("fused AdamW unavailable (%s); falling back to foreach", e)
+        return torch.optim.AdamW(groups, **kw)
+
+
+class Trainer:
+    """Minimal MLM step machinery shared by the finder, the throughput run and the
+    convergence check, so all three measure the same code path the real trainer will use."""
+
+    def __init__(self, model: EncoderForMLM, opt: torch.optim.AdamW, pool: TokenPool,
+                 cfg: dict, mcfg: EncoderConfig, gen: torch.Generator) -> None:
+        self.model, self.opt, self.pool, self.gen = model, opt, pool, gen
+        self.vocab = mcfg.vocab_size
+        p = cfg["pretrain"]
+        self.p_mask = p["mlm_probability"]
+        self.replace_probs = list(p["mask_replace_probs"])
+        self.grad_clip = p["grad_clip"]
+
+    def fwd_bwd(self, micro_batch: int, loss_scale: float = 1.0) -> float:
+        ids = self.pool.batch(micro_batch, self.gen)
+        inputs, labels, pos = mlm_mask(ids, self.pool, self.vocab, self.p_mask, self.replace_probs, self.gen)
+        with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+            logits = self.model(inputs, attention_mask=None, masked_positions=pos)
+            loss = F.cross_entropy(logits, labels)
+        (loss * loss_scale).backward()
+        return float(loss.detach())
+
+    def opt_step(self) -> None:
+        torch.nn.utils.clip_grad_norm_(self.model.parameters(), self.grad_clip)
+        self.opt.step()
+        self.opt.zero_grad(set_to_none=True)
+
+    def step(self, micro_batch: int) -> float:
+        loss = self.fwd_bwd(micro_batch)
+        self.opt_step()
+        return loss
+
+    def accum_step(self, micro_batch: int, accum: int) -> float:
+        """One optimizer step from ``accum`` accumulated micro-steps: the production
+        pattern. From the second micro-step on, the fp32 ``.grad`` tensors of the whole
+        model (4 B/param = 1.26 GiB) are resident while a fresh activation set is built,
+        so this is the pattern that exposes the true peak; a lone fwd/bwd/opt step
+        measures one gradient copy too little. Returns the mean micro-step loss."""
+        total = 0.0
+        for _ in range(accum):
+            total += self.fwd_bwd(micro_batch, 1.0 / accum)
+        self.opt_step()
+        return total / accum
+
+    def release(self) -> None:
+        self.opt.zero_grad(set_to_none=True)
+        gc.collect()
+        torch.cuda.empty_cache()
+
+
+# ------------------------------------------------------------------------- step 4: finder
+@dataclass
+class ProbeResult:
+    micro_batch: int
+    grad_checkpointing: bool
+    ok: bool
+    steps: int                   # timed optimizer steps, each of `micro_steps` accumulated fwd/bwd
+    micro_steps: int
+    max_allocated_gib: float
+    max_reserved_gib: float
+    device_used_after_gib: float
+    ms_per_micro_step: float | None   # includes 1/micro_steps of an optimizer step: rough, see throughput
+    tokens_per_s: float | None
+    error: str | None
+
+
+def probe(trainer: Trainer, micro_batch: int, steps: int, seq_len: int, ckpt: bool) -> ProbeResult:
+    trainer.model.config.grad_checkpointing = ckpt
+    trainer.release()
+    torch.cuda.synchronize()
+    torch.cuda.reset_peak_memory_stats()
+    ok, err, ms = True, None, None
+    try:
+        # One untimed step absorbs cuBLAS/allocator warmup at this shape and allocates the
+        # AdamW state; the peak-memory window covers it too. Every step uses the
+        # accumulation pattern so the resident-gradient peak is part of the measurement.
+        trainer.accum_step(micro_batch, PROBE_ACCUM)
+        torch.cuda.synchronize()
+        t0 = common.timer()
+        for _ in range(steps):
+            trainer.accum_step(micro_batch, PROBE_ACCUM)
+        torch.cuda.synchronize()
+        ms = (common.timer() - t0) * 1000.0 / (steps * PROBE_ACCUM)
+    except torch.OutOfMemoryError as e:
+        ok = False
+        err = str(e).splitlines()[0][:240]
+        LOG.warning("B=%d ckpt=%s: OOM -> %s", micro_batch, ckpt, err)
+    peak_alloc = torch.cuda.max_memory_allocated() / GIB
+    peak_res = torch.cuda.max_memory_reserved() / GIB
+    free, total = torch.cuda.mem_get_info()
+    trainer.release()
+    tok_s = micro_batch * seq_len / (ms / 1000.0) if ms else None
+    r = ProbeResult(micro_batch, ckpt, ok, steps, PROBE_ACCUM, round(peak_alloc, 3), round(peak_res, 3),
+                    round((total - free) / GIB, 3), round(ms, 1) if ms else None,
+                    round(tok_s) if tok_s else None, err)
+    LOG.info("probe B=%d ckpt=%s ok=%s peak_alloc=%.2f GiB peak_reserved=%.2f GiB ms/micro-step=%s tok/s=%s",
+             r.micro_batch, ckpt, ok, peak_alloc, peak_res, r.ms_per_micro_step, r.tokens_per_s)
+    return r
+
+
+def choose_micro_batch(results: list[ProbeResult], low: float, high: float, cap: float) -> tuple[int | None, str]:
+    fit = [r for r in results if r.ok and r.max_reserved_gib <= cap]
+    in_band = [r for r in fit if low <= r.max_reserved_gib <= high]
+    if in_band:
+        b = max(in_band, key=lambda r: r.micro_batch)
+        return b.micro_batch, f"largest candidate with reserved peak in [{low}, {high}] GiB ({b.max_reserved_gib} GiB)"
+    under = [r for r in fit if r.max_reserved_gib <= high]
+    if under:
+        b = max(under, key=lambda r: r.micro_batch)
+        return b.micro_batch, f"no candidate reached {low} GiB; largest under {high} GiB ({b.max_reserved_gib} GiB)"
+    if fit:
+        b = min(fit, key=lambda r: r.micro_batch)
+        return b.micro_batch, f"WARNING every fitting candidate exceeds {high} GiB; smallest under the {cap} GiB cap ({b.max_reserved_gib} GiB)"
+    return None, "no candidate fits under the hard cap"
+
+
+# --------------------------------------------------------------------- step 5: throughput
+def measure_throughput(trainer: Trainer, micro_batch: int, seq_len: int, eff_batch: int,
+                       timed_steps: int, warmup_steps: int, epoch_tokens: float) -> dict[str, Any]:
+    trainer.model.config.grad_checkpointing = False
+    trainer.release()
+    torch.cuda.synchronize()
+    torch.cuda.reset_peak_memory_stats()
+    # Nearest integer, not floor: 256 is divisible by neither 24 nor 12, and every number
+    # below must describe the batch that is actually realised (floor would silently train
+    # at 240 while reporting 256). The realised batch is reported next to the target.
+    accum = max(1, round(eff_batch / micro_batch))
+    eff_actual = accum * micro_batch
+    if eff_actual != eff_batch:
+        LOG.warning("effective batch %d (= %d x %d) differs from the configured %d because %d is not a divisor; "
+                    "tokens/s, steps/epoch and hours below use the realised %d",
+                    eff_actual, accum, micro_batch, eff_batch, micro_batch, eff_actual)
+    scale = 1.0 / accum
+
+    def micro(i: int) -> None:
+        trainer.fwd_bwd(micro_batch, scale)
+        if (i + 1) % accum == 0:
+            trainer.opt_step()
+
+    for i in range(warmup_steps):
+        micro(i)
+    trainer.release()
+    torch.cuda.synchronize()
+    start, end = torch.cuda.Event(enable_timing=True), torch.cuda.Event(enable_timing=True)
+    start.record()
+    for i in range(timed_steps):
+        micro(i)
+    end.record()
+    torch.cuda.synchronize()
+    elapsed_ms = start.elapsed_time(end)
+    opt_steps_in_window = timed_steps // accum
+    trainer.release()
+
+    # Isolated optimizer-step cost (clip + fused AdamW + zero_grad), amortised over `accum`.
+    opt_ms = []
+    for _ in range(3):
+        trainer.fwd_bwd(micro_batch, scale)
+        torch.cuda.synchronize()
+        s, e = torch.cuda.Event(enable_timing=True), torch.cuda.Event(enable_timing=True)
+        s.record()
+        trainer.opt_step()
+        e.record()
+        torch.cuda.synchronize()
+        opt_ms.append(s.elapsed_time(e))
+    trainer.release()
+    ms_opt = sum(opt_ms) / len(opt_ms)
+
+    ms_fwd_bwd = (elapsed_ms - opt_steps_in_window * ms_opt) / timed_steps
+    ms_micro_raw = elapsed_ms / timed_steps
+    tokens_per_micro = micro_batch * seq_len
+    tok_s_raw = tokens_per_micro / (ms_micro_raw / 1000.0)
+    ms_per_opt_step = accum * ms_fwd_bwd + ms_opt
+    tok_s_eff = eff_actual * seq_len / (ms_per_opt_step / 1000.0)
+    steps_per_epoch = epoch_tokens / (eff_actual * seq_len)
+    hours_epoch = epoch_tokens / tok_s_eff / 3600.0
+    vram = common.vram_stats()
+    out = {
+        "micro_batch": micro_batch, "grad_accumulation": accum, "effective_batch": eff_actual,
+        "effective_batch_target": eff_batch, "effective_batch_matches_target": eff_actual == eff_batch,
+        "peak_reserved_gib": round(vram["max_reserved_gb"], 3), "peak_allocated_gib": round(vram["max_allocated_gb"], 3),
+        "timed_micro_steps": timed_steps, "warmup_micro_steps": warmup_steps,
+        "ms_per_micro_step_raw": round(ms_micro_raw, 2), "ms_fwd_bwd": round(ms_fwd_bwd, 2),
+        "ms_optimizer_step": round(ms_opt, 2), "ms_per_optimizer_step_total": round(ms_per_opt_step, 1),
+        "tokens_per_s_raw": round(tok_s_raw), "tokens_per_s_effective": round(tok_s_eff),
+        "epoch_tokens": epoch_tokens, "optimizer_steps_per_epoch": round(steps_per_epoch),
+        "hours_per_epoch": round(hours_epoch, 2), "hours_3_epochs": round(3 * hours_epoch, 2),
+        "vram": vram,
+    }
+    LOG.info("throughput: %s", out)
+    return out
+
+
+def resolve_epoch_tokens(override: float | None, pool: TokenPool, processed: Path) -> tuple[float, str]:
+    """Tokens per pretraining epoch for the ETA: an explicit --epoch-tokens wins; else the
+    data manifest's pretraining character count scaled by the tokens/char measured on the
+    packed sample (the only place the real tokenizer and the real corpus meet); else the
+    research estimate."""
+    if override is not None:
+        return float(override), "--epoch-tokens"
+    manifest = processed / "splits_manifest.json"
+    if pool.tokens_per_char and manifest.exists():
+        m = common.read_json(manifest)
+        chars = int(m["counts"]["totals"]["pretrain"]["chars"])
+        note = " [dev subsample]" if m.get("dev_subsample") else ""
+        return chars * pool.tokens_per_char, f"splits_manifest.json {chars:,} chars x {pool.tokens_per_char:.4f} tok/char{note}"
+    return DEFAULT_EPOCH_TOKENS, "verify_data.md estimate (117M tokens at 4 chars/token)"
+
+
+def check_sdpa_backends(model: EncoderForMLM, seq_len: int, vocab: int) -> dict[str, Any]:
+    """Which fused kernel actually runs. Flash must take the mask-free (packed) path and
+    memory-efficient the bool-mask path; a silent math fallback would store the LxL
+    softmax and blow the activation budget."""
+    model.config.grad_checkpointing = False
+    ids = torch.randint(5, vocab, (2, seq_len), device="cuda")
+    mask = torch.ones(2, seq_len, dtype=torch.bool, device="cuda")
+    mask[1, seq_len // 2:] = False
+    cases = {
+        "flash_no_mask": ([SDPBackend.FLASH_ATTENTION], None),
+        "efficient_bool_mask": ([SDPBackend.EFFICIENT_ATTENTION], mask),
+        "flash_bool_mask_expected_fail": ([SDPBackend.FLASH_ATTENTION], mask),
+    }
+    out: dict[str, Any] = {}
+    for name, (backends, m) in cases.items():
+        try:
+            with warnings.catch_warnings():
+                warnings.simplefilter("ignore")
+                with sdpa_kernel(backends), torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+                    h = model.encoder(ids, m)
+                h.float().mean().backward()
+            out[name] = "ran"
+        except RuntimeError as e:
+            out[name] = f"unavailable: {str(e).splitlines()[0][:100]}"
+        model.zero_grad(set_to_none=True)
+    out["flags"] = {
+        "flash_sdp_enabled": torch.backends.cuda.flash_sdp_enabled(),
+        "mem_efficient_sdp_enabled": torch.backends.cuda.mem_efficient_sdp_enabled(),
+        "math_sdp_enabled": torch.backends.cuda.math_sdp_enabled(),
+    }
+    LOG.info("sdpa backends: %s", out)
+    return out
+
+
+# -------------------------------------------------------------------- step 6: convergence
+def convergence_check(cfg: dict, mcfg: EncoderConfig, pool: TokenPool, device: torch.device,
+                      micro_batch: int, steps: int, lr: float, warmup: int) -> dict[str, Any]:
+    model = build_model(mcfg, device, cfg["seed"] + 1)
+    opt = build_optimizer(model, lr, cfg["pretrain"])
+    gen = torch.Generator(device=device).manual_seed(cfg["seed"] + 1)
+    tr = Trainer(model, opt, pool, cfg, mcfg, gen)
+    losses: list[tuple[int, float]] = []
+    for step in range(1, steps + 1):
+        for g in opt.param_groups:
+            g["lr"] = lr * min(1.0, step / max(1, warmup))
+        loss = tr.step(micro_batch)
+        if not math.isfinite(loss):
+            LOG.error("non-finite loss at step %d", step)
+            losses.append((step, loss))
+            break
+        if step == 1 or step % 5 == 0 or step == steps:
+            losses.append((step, round(loss, 4)))
+            LOG.info("convergence step %3d/%d  loss %.4f  lr %.2e", step, steps, loss, opt.param_groups[0]["lr"])
+    tr.release()
+    first, last = losses[0][1], losses[-1][1]
+    out = {
+        "micro_batch": micro_batch, "steps": steps, "lr": lr, "warmup_steps": warmup,
+        "ln_vocab": round(math.log(mcfg.vocab_size), 4), "first_loss": first, "last_loss": last,
+        "drop": round(first - last, 4), "losses": losses, "data": pool.source,
+        "vram": common.vram_stats(),
+    }
+    del tr, opt, model
+    gc.collect()
+    torch.cuda.empty_cache()
+    return out
+
+
+# ------------------------------------------------------------------------ step 7: latency
+def _time_gpu(fn, n: int, warmup: int) -> dict[str, float]:
+    for _ in range(warmup):
+        fn()
+    torch.cuda.synchronize()
+    times = []
+    for _ in range(n):
+        s, e = torch.cuda.Event(enable_timing=True), torch.cuda.Event(enable_timing=True)
+        s.record()
+        fn()
+        e.record()
+        torch.cuda.synchronize()
+        times.append(s.elapsed_time(e))
+    times.sort()
+    return {"n": n, "avg_ms": round(sum(times) / n, 2), "p50_ms": round(times[n // 2], 2), "max_ms": round(times[-1], 2)}
+
+
+def _time_cpu(fn, n: int, warmup: int) -> dict[str, float]:
+    for _ in range(warmup):
+        fn()
+    times = []
+    for _ in range(n):
+        t0 = time.perf_counter()
+        fn()
+        times.append((time.perf_counter() - t0) * 1000.0)
+    times.sort()
+    return {"n": n, "avg_ms": round(sum(times) / n, 1), "p50_ms": round(times[n // 2], 1), "max_ms": round(times[-1], 1)}
+
+
+def inference_latency(cfg: dict, mcfg: EncoderConfig, device: torch.device, seq_len: int,
+                      gpu_samples: int, cpu_samples: int, cpu_threads: int, skip_cpu: bool) -> dict[str, Any]:
+    common.set_seed(cfg["seed"])
+    with torch.device(device):
+        clf = EncoderForClassification(mcfg)
+    clf.eval()
+    out: dict[str, Any] = {"instructor_benchmark_ms": INSTRUCTOR_BENCHMARK_S * 1000.0, "params": count_parameters(clf)["total"]}
+    ids = torch.randint(5, mcfg.vocab_size, (1, seq_len), device=device)
+    full_mask = torch.ones(1, seq_len, dtype=torch.bool, device=device)
+    with torch.inference_mode(), torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+        out[f"gpu_bf16_{seq_len}_no_mask"] = _time_gpu(lambda: clf(ids), gpu_samples, 10)
+        out[f"gpu_bf16_{seq_len}_bool_mask"] = _time_gpu(lambda: clf(ids, full_mask), gpu_samples, 10)
+    LOG.info("GPU latency: %s", {k: v for k, v in out.items() if k.startswith("gpu")})
+    if not skip_cpu:
+        clf = clf.cpu().float()
+        torch.set_num_threads(cpu_threads)
+        out["cpu_threads"] = torch.get_num_threads()
+        with torch.inference_mode():
+            for L in (seq_len, 128):
+                x = ids[:, :L].cpu()
+                out[f"cpu_fp32_{L}"] = _time_cpu(lambda: clf(x), cpu_samples, 2)
+                LOG.info("CPU fp32 L=%d: %s", L, out[f"cpu_fp32_{L}"])
+    del clf
+    gc.collect()
+    torch.cuda.empty_cache()
+    return out
+
+
+# ------------------------------------------------------------------------------ report
+def _probe_table(rows: list[ProbeResult]) -> str:
+    head = "| micro-batch | ckpt | result | peak alloc GiB | peak reserved GiB | device used GiB | ms/micro-step | tok/s |\n|---|---|---|---|---|---|---|---|"
+    lines = [head]
+    for r in rows:
+        res = "ok" if r.ok else "OOM"
+        lines.append(f"| {r.micro_batch} | {'on' if r.grad_checkpointing else 'off'} | {res} | {r.max_allocated_gib:.2f} | "
+                     f"{r.max_reserved_gib:.2f} | {r.device_used_after_gib:.2f} | {r.ms_per_micro_step or '-'} | "
+                     f"{f'{r.tokens_per_s:,}' if r.tokens_per_s else '-'} |")
+    return "\n".join(lines)
+
+
+def markdown_summary(rep: dict[str, Any]) -> str:
+    d = rep["device"]
+    vr = rep["vram_policy"]
+    lines = [
+        "## Smoke test summary",
+        f"- device: {d.get('device', 'cpu')} (sm {d.get('sm', '?')}, {d.get('vram_gib', '?')} GiB, bf16={d.get('bf16', '?')}); "
+        f"torch {d.get('torch')} / python {d.get('python')}; device used before model build: {rep['baseline_device_used_gib']} GiB",
+        f"- model: {rep['param_counts']['mlm']['total']:,} params (MLM, tied) / {rep['param_counts']['classifier']['total']:,} (classifier)",
+        f"- data: {rep['data']['source']} -- {rep['data']['detail']}",
+        f"- VRAM policy: target [{vr['target_gb_low']}, {vr['target_gb_high']}] GiB reserved, hard cap {vr['hard_cap_gb']} GiB, alloc_conf={vr['alloc_conf']} ({VRAM_UNIT_NOTE})",
+        "", f"### Micro-batch finder (no checkpointing; {rep['finder']['probe_steps']} timed optimizer steps of "
+            f"{PROBE_ACCUM} accumulated micro-steps after 1 warmup step per candidate; grads resident during the 2nd micro-step)",
+        _probe_table([ProbeResult(**r) for r in rep['finder']['no_ckpt']]),
+        f"\n**chosen micro-batch: {rep['finder']['chosen_micro_batch']}** -- {rep['finder']['reason']}",
+    ]
+    fin = rep["finder"]
+    if fin.get("throughput_peak_reserved_gib") is not None:
+        verdict = ("WITHIN the hard cap" if fin["throughput_within_hard_cap"] else "**BREACHES the hard cap**")
+        band = "in the target band" if fin["throughput_in_target_band"] else "outside the target band"
+        lines.append(f"- second opinion from the throughput run (real accumulation at the chosen batch): "
+                     f"{fin['throughput_peak_reserved_gib']} GiB reserved peak -- {verdict}, {band}")
+    lines += [
+        "", "### Micro-batch finder (gradient checkpointing on, ~x1.33 compute)", _probe_table([ProbeResult(**r) for r in rep['finder']['ckpt']]),
+        f"\ncheckpointing choice: {rep['finder']['chosen_micro_batch_ckpt']} -- {rep['finder']['reason_ckpt']}",
+    ]
+    t = rep.get("throughput")
+    if t:
+        eff_note = "" if t["effective_batch_matches_target"] else f" (**target {t['effective_batch_target']} not divisible by {t['micro_batch']}; all numbers use {t['effective_batch']}**)"
+        lines += [
+            "", "### Throughput (chosen micro-batch, no checkpointing)",
+            f"- micro-batch {t['micro_batch']} x accumulation {t['grad_accumulation']} = effective {t['effective_batch']} sequences{eff_note}",
+            f"- fwd+bwd {t['ms_fwd_bwd']} ms/micro-step, optimizer step {t['ms_optimizer_step']} ms -> {t['ms_per_optimizer_step_total']} ms per optimizer step",
+            f"- **{t['tokens_per_s_effective']:,} tokens/s** effective ({t['tokens_per_s_raw']:,} raw over {t['timed_micro_steps']} timed micro-steps); peak reserved {t['peak_reserved_gib']} GiB",
+            f"- {t['epoch_tokens']/1e6:.1f}M-token epoch ({t.get('epoch_tokens_source', '?')}) = {t['optimizer_steps_per_epoch']} optimizer steps -> "
+            f"**{t['hours_per_epoch']} h/epoch, {t['hours_3_epochs']} h for 3 epochs**",
+        ]
+    s = rep.get("sdpa_backends")
+    if s:
+        lines += ["", "### SDPA backends", *(f"- {k}: {v}" for k, v in s.items())]
+    c = rep.get("convergence")
+    if c:
+        lines += [
+            "", "### Mini convergence check",
+            f"- {c['steps']} steps at micro-batch {c['micro_batch']}, LR {c['lr']:.0e} with {c['warmup_steps']}-step warmup, data={c['data']}",
+            f"- ln(V) = {c['ln_vocab']}; first loss {c['first_loss']}, last loss {c['last_loss']} (drop {c['drop']})",
+            "- trace: " + ", ".join(f"{st}:{lo}" for st, lo in c["losses"]),
+        ]
+    lat = rep.get("latency")
+    if lat:
+        lines += ["", "### Inference latency (batch 1), instructor benchmark 74 ms/sample",
+                  "| setting | n | avg ms | p50 ms | max ms | vs 74 ms |", "|---|---|---|---|---|---|"]
+        for k, v in lat.items():
+            if isinstance(v, dict) and "avg_ms" in v:
+                ratio = v["avg_ms"] / lat["instructor_benchmark_ms"]
+                lines.append(f"| {k} | {v['n']} | {v['avg_ms']} | {v['p50_ms']} | {v['max_ms']} | {ratio:.2f}x |")
+    lines += ["", f"total runtime: {rep['runtime_s']} s; report: {rep['report_path']}"]
+    return "\n".join(lines)
+
+
+# -------------------------------------------------------------------------------- main
+def parse_args(argv: list[str] | None = None) -> argparse.Namespace:
+    ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    ap.add_argument("--config", default="config.yaml")
+    ap.add_argument("--steps", type=int, default=30, help="optimizer steps for the convergence check")
+    ap.add_argument("--skip-cpu", action="store_true", help="skip the CPU fp32 latency samples")
+    ap.add_argument("--candidates", default=None,
+                    help="comma list of micro-batches to probe (default: config pretrain.micro_batch_candidates "
+                         "if present, else micro_batch_start,24,12,8,4)")
+    ap.add_argument("--ckpt-candidates", default=None, help="micro-batches for the checkpointing sweep (default: same)")
+    ap.add_argument("--probe-steps", type=int, default=3)
+    ap.add_argument("--timed-steps", type=int, default=20)
+    ap.add_argument("--n-docs", type=int, default=3000, help="real documents to pack when the parquet exists")
+    ap.add_argument("--random-seqs", type=int, default=1024, help="random sequences when no real data exists")
+    ap.add_argument("--epoch-tokens", type=float, default=None,
+                    help="tokens per pretraining epoch for the ETA (default: data manifest chars x measured tokens/char, else 117e6)")
+    ap.add_argument("--conv-lr", type=float, default=1e-4)
+    ap.add_argument("--conv-warmup", type=int, default=10)
+    ap.add_argument("--cpu-threads", type=int, default=16)
+    ap.add_argument("--latency-samples", type=int, default=None, help="GPU latency samples (default config.eval.latency_samples)")
+    return ap.parse_args(argv)
+
+
+def main(argv: list[str] | None = None) -> int:
+    args = parse_args(argv)
+    t_start = common.timer()
+    cfg = common.load_config(args.config)
+    logger = common.setup_logging("smoke_test", cfg["paths"]["logs"])
+    global LOG
+    LOG = logger
+    device_info = common.print_device_info()
+    if not torch.cuda.is_available():
+        LOG.error("CUDA is required for the smoke test")
+        return 2
+    LOG.info("PYTORCH_CUDA_ALLOC_CONF=%s", os.environ.get("PYTORCH_CUDA_ALLOC_CONF"))
+    common.set_seed(cfg["seed"])
+    device = torch.device("cuda")
+    mcfg = EncoderConfig.from_dict(cfg["model"])
+    L = mcfg.max_seq_len
+    vr = cfg["vram"]
+    pcfg = cfg["pretrain"]
+    free, total = torch.cuda.mem_get_info()
+    baseline_used = round((total - free) / GIB, 2)
+    LOG.info("device used before model build (desktop + others): %.2f GiB", baseline_used)
+
+    # 2. model
+    model = build_model(mcfg, device, cfg["seed"])
+    mlm_counts = count_parameters(model)
+    LOG.info("EncoderForMLM params: %s", mlm_counts)
+    print(f"EncoderForMLM: total {mlm_counts['total']:,} | embeddings {mlm_counts['embeddings']:,} | "
+          f"layers {mlm_counts['encoder_layers']:,} | final_norm {mlm_counts['final_norm']:,} | head {mlm_counts['head']:,}")
+    with torch.device("meta"):
+        cls_counts = count_parameters(EncoderForClassification(mcfg))
+    LOG.info("EncoderForClassification params: %s", cls_counts)
+    LOG.info("after model build: %s", common.vram_stats())
+
+    # 3. data
+    pool = build_token_pool(cfg, mcfg, device, args.n_docs, args.random_seqs)
+    LOG.info("token pool: %s -> %s", pool.source, pool.detail)
+
+    # 4. finder
+    opt = build_optimizer(model, pcfg["lr"], pcfg)
+    gen = torch.Generator(device=device).manual_seed(cfg["seed"])
+    trainer = Trainer(model, opt, pool, cfg, mcfg, gen)
+    start = int(pcfg["micro_batch_start"])
+    default_cands = ([int(b) for b in pcfg["micro_batch_candidates"]] if pcfg.get("micro_batch_candidates")
+                     else [start] + [b for b in (24, 12, 8, 4) if b != start])
+    cands = [int(x) for x in args.candidates.split(",")] if args.candidates else default_cands
+    ckpt_cands = [int(x) for x in args.ckpt_candidates.split(",")] if args.ckpt_candidates else cands
+    cands = [b for b in cands if b <= pool.n] or [pool.n]
+    ckpt_cands = [b for b in ckpt_cands if b <= pool.n] or [pool.n]
+    LOG.info("finder candidates: %s (no ckpt), %s (ckpt)", cands, ckpt_cands)
+    # probe() flips grad_checkpointing on model.config, which IS mcfg (same dataclass
+    # instance); remember the config.yaml value so later model builds inherit it, not the
+    # last probe's setting.
+    ckpt_default = bool(mcfg.grad_checkpointing)
+    no_ckpt = [probe(trainer, b, args.probe_steps, L, ckpt=False) for b in cands]
+    chosen, reason = choose_micro_batch(no_ckpt, vr["target_gb_low"], vr["target_gb_high"], vr["hard_cap_gb"])
+    LOG.info("chosen micro-batch (no ckpt): %s -- %s", chosen, reason)
+    with_ckpt = [probe(trainer, b, args.probe_steps, L, ckpt=True) for b in ckpt_cands]
+    chosen_ckpt, reason_ckpt = choose_micro_batch(with_ckpt, vr["target_gb_low"], vr["target_gb_high"], vr["hard_cap_gb"])
+    LOG.info("chosen micro-batch (ckpt): %s -- %s", chosen_ckpt, reason_ckpt)
+    mcfg.grad_checkpointing = ckpt_default
+
+    report: dict[str, Any] = {
+        "timestamp": time.strftime("%Y-%m-%d %H:%M:%S"),
+        "device": device_info,
+        "alloc_conf": os.environ.get("PYTORCH_CUDA_ALLOC_CONF"),
+        "baseline_device_used_gib": baseline_used,
+        "model_config": mcfg.to_dict(),
+        "param_counts": {"mlm": mlm_counts, "classifier": cls_counts},
+        "data": {"source": pool.source, "detail": pool.detail, "n_sequences": pool.n, "seq_len": L,
+                 "mlm_probability": pcfg["mlm_probability"], "mask_replace_probs": pcfg["mask_replace_probs"]},
+        "vram_policy": vr,
+        "vram_policy_unit": VRAM_UNIT_NOTE,
+        "finder": {"candidates": cands, "ckpt_candidates": ckpt_cands, "probe_steps": args.probe_steps,
+                   "micro_steps_per_probe_step": PROBE_ACCUM,
+                   "no_ckpt": [asdict(r) for r in no_ckpt], "ckpt": [asdict(r) for r in with_ckpt],
+                   "chosen_micro_batch": chosen, "reason": reason,
+                   "chosen_micro_batch_ckpt": chosen_ckpt, "reason_ckpt": reason_ckpt},
+    }
+
+    if chosen is None:
+        LOG.error("no micro-batch fits; skipping throughput and convergence")
+    else:
+        # 5. throughput + backend check
+        epoch_tokens, epoch_src = resolve_epoch_tokens(args.epoch_tokens, pool, Path(cfg["paths"]["data_processed"]))
+        LOG.info("epoch tokens for the ETA: %.3g (%s)", epoch_tokens, epoch_src)
+        tp = measure_throughput(trainer, chosen, L, int(pcfg["effective_batch_sequences"]), args.timed_steps, 3, epoch_tokens)
+        tp["epoch_tokens_source"] = epoch_src
+        report["throughput"] = tp
+        # The throughput run is the chosen batch under the real accumulation pattern over
+        # many more micro-steps than the probe; its reserved peak is the second opinion on
+        # the finder's decision against the same cap and band.
+        fin = report["finder"]
+        fin["throughput_peak_reserved_gib"] = tp["peak_reserved_gib"]
+        fin["throughput_within_hard_cap"] = tp["peak_reserved_gib"] <= vr["hard_cap_gb"]
+        fin["throughput_in_target_band"] = vr["target_gb_low"] <= tp["peak_reserved_gib"] <= vr["target_gb_high"]
+        if not fin["throughput_within_hard_cap"]:
+            LOG.error("micro-batch %d breaches the %s GiB hard cap under real accumulation (%.2f GiB reserved); "
+                      "do not train at this batch", chosen, vr["hard_cap_gb"], tp["peak_reserved_gib"])
+        elif not fin["throughput_in_target_band"]:
+            LOG.warning("micro-batch %d: %.2f GiB reserved under real accumulation is outside the [%s, %s] GiB band",
+                        chosen, tp["peak_reserved_gib"], vr["target_gb_low"], vr["target_gb_high"])
+        report["sdpa_backends"] = check_sdpa_backends(model, L, mcfg.vocab_size)
+
+    trainer.release()
+    del trainer, opt, model
+    gc.collect()
+    torch.cuda.empty_cache()
+
+    if chosen is not None:
+        # 6. convergence from a fresh init
+        report["convergence"] = convergence_check(cfg, mcfg, pool, device, chosen, args.steps, args.conv_lr, args.conv_warmup)
+
+    # 7. latency
+    del pool
+    gc.collect()
+    torch.cuda.empty_cache()
+    n_lat = args.latency_samples or int(cfg["eval"]["latency_samples"])
+    report["latency"] = inference_latency(cfg, mcfg, device, L, n_lat, 10, args.cpu_threads, args.skip_cpu)
+
+    # 8. report
+    report["runtime_s"] = round(common.timer() - t_start, 1)
+    report_path = Path(cfg["paths"]["reports"]) / "smoke_test.json"
+    report["report_path"] = str(report_path)
+    common.write_json(report, report_path)
+    LOG.info("wrote %s (%.1f s total)", report_path, report["runtime_s"])
+    print()
+    print(markdown_summary(report))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/presentation/src/tokenizer.py b/presentation/src/tokenizer.py
new file mode 100644
index 0000000000000000000000000000000000000000..f24e143abffcac90641032ca5cbc39b09d58836e
--- /dev/null
+++ b/presentation/src/tokenizer.py
@@ -0,0 +1,449 @@
+"""Byte-level BPE tokenizer for the phishing encoder: training, statistics, and batch encoding.
+
+Design (config.tokenizer):
+- NFKC normalisation only, case preserved. ALL-CAPS subjects ("URGENT", "VERIFY YOUR ACCOUNT"),
+  odd casing in brand names and shouty pharma spam are strong phishing/spam signals; lowercasing
+  would throw them away. NFKC folds full-width and ligature look-alikes (a common filter-evasion
+  trick) onto their ASCII forms so they share subwords with the plain spelling.
+- Byte-level pre-tokenizer with the full 256-symbol initial alphabet: every byte sequence is
+  representable, so [UNK] is unreachable in practice. Email is full of URLs, base64 fragments,
+  HTML entities and mojibake, which a character-level vocabulary would map to [UNK].
+- add_prefix_space=False: inputs are "subject\\n\\nbody", not isolated words.
+- 32,768 ids (2^15) to match model.vocab_size; [PAD] is id 0 because the model's embedding uses
+  padding_idx=0 and the collate code pads with 0.
+- Trained ONLY on pretrain.parquet + finetune_train.parquet. finetune_val and test_holdout
+  (TREC-06) never touch the vocabulary: a vocabulary fitted on test text leaks its frequent
+  strings (template URLs, spam phrases) into single tokens.
+"""
+
+from __future__ import annotations
+
+import argparse
+import logging
+import sys
+from collections.abc import Iterator, Sequence
+from dataclasses import asdict, dataclass, field
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).parent))
+
+import numpy as np
+import pyarrow.parquet as pq
+import torch
+from tokenizers import Tokenizer, decoders, models, normalizers, pre_tokenizers, processors, trainers
+
+import common
+
+LOG = logging.getLogger("tokenizer")
+
+SPLIT_FILES: dict[str, str] = {
+    "pretrain": "pretrain.parquet",
+    "finetune_train": "finetune_train.parquet",
+    "finetune_val": "finetune_val.parquet",
+    "test_holdout": "test_holdout.parquet",
+}
+FORBIDDEN_TRAIN_FILES = frozenset({"finetune_val.parquet", "test_holdout.parquet"})
+SOURCE_COLUMNS = ("source", "dataset_name")
+TEXT_COLUMN = "text"
+READ_BATCH_ROWS = 2048
+STATS_ENCODE_BATCH = 4096
+PRETRAIN_STATS_SAMPLE = 20_000
+
+EXAMPLES: list[tuple[str, str]] = [
+    ("business", "Re: Q3 budget review\n\nHi Sarah, can we move the budget review to Thursday at 2pm? "
+                 "I've attached the revised forecast. Thanks, Mike"),
+    ("url_heavy", "Your account has been suspended\n\nVerify now: http://secure-login.paypa1-verify.com/"
+                  "account/update?id=8f3a9c2e&session=XyZ123 or https://bit.ly/3xYz9Ab before 24 hours."),
+    ("all_caps", "URGENT!!! FINAL NOTICE\n\nYOUR MAILBOX IS FULL. CLICK HERE TO UPGRADE YOUR STORAGE "
+                 "IMMEDIATELY OR LOSE ALL MESSAGES."),
+    ("unicode_lookalike", "Ｍｉｃｒｏｓｏｆｔ 365 — Password expiry ⚠️\n\nDear user, your pass­word expires "
+                          "today. Café, naïve, 東京 office."),
+    ("html_money", "<html><body><p>Dear Friend,</p><p>I am Barrister James. Transfer of US$15,500,000.00 "
+                   "&amp; 30% for you.</p></body></html>"),
+]
+
+
+@dataclass
+class TokenizerPaths:
+    data_dir: Path
+    tokenizer_json: Path
+    report_json: Path
+    log_dir: Path
+
+
+@dataclass
+class SplitStats:
+    split: str
+    file: str
+    rows: int
+    mean_tokens: float
+    median_tokens: float
+    p95_tokens: float
+    max_tokens: int
+    chars_per_token: float
+    pct_truncated: float
+    unk_rate: float
+    mean_tokens_by_label: dict[str, float] = field(default_factory=dict)
+
+
+def _abs(p: str | Path) -> Path:
+    p = Path(p)
+    return p if p.is_absolute() else common.project_root() / p
+
+
+def resolve_paths(cfg: dict, data_dir: str | None) -> TokenizerPaths:
+    """--data-dir redirects data, artifacts and logs together, so a self-test never writes into the project."""
+    if data_dir is not None:
+        d = Path(data_dir).resolve()
+        return TokenizerPaths(d, d / "tokenizer.json", d / "tokenizer_report.json", d / "logs")
+    d = _abs(cfg["paths"]["data_processed"])
+    tok = _abs(cfg["paths"]["tokenizer"])
+    return TokenizerPaths(d, tok, tok.with_name("tokenizer_report.json"), _abs(cfg["paths"]["logs"]))
+
+
+# ---------------------------------------------------------------------------------------- leakage
+
+
+def assert_trainable(path: Path, holdout_source: str) -> None:
+    """Refuse validation/test files by name, and any file whose source column carries the holdout corpus."""
+    if path.name in FORBIDDEN_TRAIN_FILES:
+        raise RuntimeError(f"refusing to train the tokenizer on {path.name}: val/test text must never shape the vocabulary")
+    schema_names = set(pq.read_schema(path).names)
+    if TEXT_COLUMN not in schema_names:
+        raise RuntimeError(f"{path} has no '{TEXT_COLUMN}' column (columns: {sorted(schema_names)})")
+    for col in SOURCE_COLUMNS:
+        if col in schema_names:
+            values = {str(v) for v in pq.read_table(path, columns=[col]).column(col).unique().to_pylist()}
+            hits = {v for v in values if v.strip().lower() == holdout_source.lower()}
+            if hits:
+                raise RuntimeError(f"{path.name} column '{col}' contains holdout source {hits}: test-set leakage")
+
+
+def iter_texts(path: Path, limit: int | None = None) -> Iterator[list[str]]:
+    """Stream the text column in record batches; the 244k-email pretraining corpus never sits in RAM twice."""
+    seen = 0
+    for batch in pq.ParquetFile(path).iter_batches(batch_size=READ_BATCH_ROWS, columns=[TEXT_COLUMN]):
+        texts = [t for t in batch.column(0).to_pylist() if t]
+        if limit is not None:
+            texts = texts[: max(0, limit - seen)]
+        if texts:
+            seen += len(texts)
+            yield texts
+        if limit is not None and seen >= limit:
+            return
+
+
+def holdout_collisions(train_files: Sequence[Path], holdout: Path, limit: int | None) -> dict[str, int]:
+    """Exact normalized-text collisions (verify_data.md §7 k_full) between training text and the holdout.
+
+    The data pipeline already decontaminates; this is an independent tripwire at the point where
+    text first becomes model state. Reading the holdout here only builds a hash set.
+    """
+    held = {common.sha1_hex(common.normalize_text(t)) for batch in iter_texts(holdout) for t in batch}
+    out: dict[str, int] = {}
+    for f in train_files:
+        out[f.name] = sum(
+            common.sha1_hex(common.normalize_text(t)) in held for batch in iter_texts(f, limit) for t in batch
+        )
+    return out
+
+
+# --------------------------------------------------------------------------------------- building
+
+
+def build_untrained() -> Tokenizer:
+    tok = Tokenizer(models.BPE(unk_token="[UNK]"))
+    tok.normalizer = normalizers.NFKC()
+    tok.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=False)
+    tok.decoder = decoders.ByteLevel()
+    return tok
+
+
+def attach_post_processor(tok: Tokenizer) -> None:
+    """BERT template; segment ids only matter for pairs, which the classifier does not use but the MLM loader may."""
+    cls_id, sep_id = tok.token_to_id("[CLS]"), tok.token_to_id("[SEP]")
+    tok.post_processor = processors.TemplateProcessing(
+        single="[CLS] $A [SEP]",
+        pair="[CLS] $A [SEP] $B:1 [SEP]:1",
+        special_tokens=[("[CLS]", cls_id), ("[SEP]", sep_id)],
+    )
+
+
+def check_special_ids(tok: Tokenizer, special_tokens: Sequence[str]) -> dict[str, int]:
+    ids = {t: tok.token_to_id(t) for t in special_tokens}
+    if ids.get("[PAD]") != 0:
+        raise AssertionError(f"[PAD] must be id 0 (model padding_idx), got {ids.get('[PAD]')}")
+    if list(ids.values()) != list(range(len(special_tokens))):
+        raise AssertionError(f"special tokens must occupy ids 0..{len(special_tokens) - 1} in order, got {ids}")
+    return ids
+
+
+def train(cfg: dict, paths: TokenizerPaths, vocab_size: int | None, limit_docs: int | None) -> Tokenizer:
+    tcfg = cfg["tokenizer"]
+    vocab_size = vocab_size or int(tcfg["vocab_size"])
+    model_vocab = int(cfg["model"]["vocab_size"])
+    if vocab_size > model_vocab:
+        raise ValueError(f"tokenizer vocab {vocab_size} exceeds model.vocab_size {model_vocab}")
+    special = list(tcfg["special_tokens"])
+    holdout_source = cfg["data"]["holdout_test_source"]
+
+    files = [paths.data_dir / SPLIT_FILES[name] for name in tcfg["train_on"]]
+    for f in files:
+        if not f.exists():
+            raise FileNotFoundError(f)
+        assert_trainable(f, holdout_source)
+
+    holdout = paths.data_dir / SPLIT_FILES["test_holdout"]
+    if holdout.exists():
+        collisions = holdout_collisions(files, holdout, limit_docs)
+        LOG.info("exact holdout collisions per training file: %s", collisions)
+        if any(collisions.values()):
+            raise RuntimeError(f"training text exactly matches TREC-06 holdout rows: {collisions}")
+    else:
+        collisions = None
+        LOG.warning("%s absent: exact-hash leakage tripwire skipped (name/source guards still applied)", holdout.name)
+
+    usage: dict[str, dict[str, int]] = {}
+
+    def corpus() -> Iterator[list[str]]:
+        for f in files:
+            rows = chars = 0
+            for batch in iter_texts(f, limit_docs):
+                rows += len(batch)
+                chars += sum(map(len, batch))
+                yield batch
+            usage[f.name] = {"rows_in_file": pq.ParquetFile(f).metadata.num_rows, "rows_used": rows, "chars": chars}
+            LOG.info("fed %s: %d rows, %d chars", f.name, rows, chars)
+
+    tok = build_untrained()
+    trainer_kwargs: dict = dict(
+        vocab_size=vocab_size,
+        min_frequency=int(tcfg["min_frequency"]),
+        special_tokens=special,
+        initial_alphabet=pre_tokenizers.ByteLevel.alphabet(),
+        show_progress=True,
+    )
+    if tcfg.get("max_token_length"):
+        trainer_kwargs["max_token_length"] = int(tcfg["max_token_length"])
+    LOG.info("training byte-level BPE: %s on %s (limit_docs=%s)",
+             {k: v for k, v in trainer_kwargs.items() if k != "initial_alphabet"}, [f.name for f in files], limit_docs)
+    t0 = common.timer()
+    tok.train_from_iterator(corpus(), trainer=trainers.BpeTrainer(**trainer_kwargs))
+    elapsed = common.timer() - t0
+
+    special_ids = check_special_ids(tok, special)
+    attach_post_processor(tok)
+    final_vocab = tok.get_vocab_size()
+    if final_vocab > model_vocab:
+        raise AssertionError(f"trained vocab {final_vocab} > model.vocab_size {model_vocab}")
+    if final_vocab < vocab_size:
+        LOG.warning("vocab reached only %d of %d (corpus too small / min_frequency); embedding rows above stay unused",
+                    final_vocab, vocab_size)
+
+    paths.tokenizer_json.parent.mkdir(parents=True, exist_ok=True)
+    tok.save(str(paths.tokenizer_json))
+    LOG.info("saved %s (vocab %d) in %.1fs", paths.tokenizer_json, final_vocab, elapsed)
+
+    report = _read_report(paths)
+    report["train"] = {
+        "tokenizer_json": str(paths.tokenizer_json),
+        "tokenizer_sha1": common.sha1_hex(paths.tokenizer_json.read_text()),
+        "requested_vocab_size": vocab_size,
+        "final_vocab_size": final_vocab,
+        "min_frequency": int(tcfg["min_frequency"]),
+        "special_token_ids": special_ids,
+        "normalizer": "NFKC (case preserved)",
+        "pre_tokenizer": "ByteLevel(add_prefix_space=False)",
+        "training_files": usage,
+        "limit_docs_per_file": limit_docs,
+        "excluded_files": sorted(FORBIDDEN_TRAIN_FILES),
+        "holdout_exact_collisions": collisions,
+        "train_seconds": round(elapsed, 1),
+    }
+    common.write_json(report, paths.report_json)
+    return tok
+
+
+# ---------------------------------------------------------------------------------------- loading
+
+
+def load_tokenizer(path: str | Path) -> Tokenizer:
+    """Load tokenizer.json with truncation/padding cleared; callers configure them (or use encode_batch)."""
+    tok = Tokenizer.from_file(str(path))
+    if tok.token_to_id("[PAD]") != 0:
+        raise AssertionError(f"{path}: [PAD] is not id 0")
+    tok.no_truncation()
+    tok.no_padding()
+    return tok
+
+
+def encode_batch(
+    tok: Tokenizer, texts: Sequence[str], max_len: int, pad_to_multiple_of: int | None = 8
+) -> tuple[torch.Tensor, torch.Tensor]:
+    """Encode with [CLS]/[SEP], truncate to max_len, pad to the longest in the batch (dynamic padding).
+
+    pad_to_multiple_of=8 keeps sequence lengths tensor-core friendly; it must divide max_len so a
+    full-length row is never padded past the model's position table.
+    Returns (input_ids int64 [B, L], attention_mask bool [B, L]).
+    """
+    if pad_to_multiple_of and max_len % pad_to_multiple_of:
+        raise ValueError(f"max_len {max_len} is not a multiple of pad_to_multiple_of {pad_to_multiple_of}")
+    if len(texts) == 0:
+        return torch.empty((0, 0), dtype=torch.int64), torch.empty((0, 0), dtype=torch.bool)
+    # Truncation/padding are configured on the shared Tokenizer object for this call only and
+    # cleared afterwards, so a later tok.encode() on the same object returns raw ids again.
+    tok.enable_truncation(max_length=max_len)
+    tok.enable_padding(pad_id=0, pad_token="[PAD]", pad_to_multiple_of=pad_to_multiple_of)
+    try:
+        encs = tok.encode_batch_fast(list(texts))
+    finally:
+        tok.no_truncation()
+        tok.no_padding()
+    ids = np.asarray([e.ids for e in encs], dtype=np.int64).reshape(len(texts), -1)
+    mask = np.asarray([e.attention_mask for e in encs], dtype=np.bool_).reshape(len(texts), -1)
+    return torch.from_numpy(ids), torch.from_numpy(mask)
+
+
+# ------------------------------------------------------------------------------------------ stats
+
+
+def _read_report(paths: TokenizerPaths) -> dict:
+    return common.read_json(paths.report_json) if paths.report_json.exists() else {}
+
+
+def _load_split(path: Path, sample: int | None, seed: int) -> tuple[list[str], list | None]:
+    cols = [TEXT_COLUMN] + (["label"] if "label" in pq.read_schema(path).names else [])
+    table = pq.read_table(path, columns=cols)
+    if sample is not None and table.num_rows > sample:
+        idx = np.sort(np.random.default_rng(seed).choice(table.num_rows, size=sample, replace=False))
+        table = table.take(idx)
+    texts = [t or "" for t in table.column(TEXT_COLUMN).to_pylist()]
+    labels = table.column("label").to_pylist() if "label" in cols else None
+    return texts, labels
+
+
+def split_stats(tok: Tokenizer, split: str, path: Path, max_len: int, sample: int | None, seed: int) -> SplitStats:
+    """Content-token counts (no specials); truncation is judged on content + [CLS] + [SEP] > max_len."""
+    texts, labels = _load_split(path, sample, seed)
+    tok.no_truncation()
+    tok.no_padding()
+    unk_id = tok.token_to_id("[UNK]")
+    lengths = np.empty(len(texts), dtype=np.int64)
+    unk = 0
+    for i in range(0, len(texts), STATS_ENCODE_BATCH):
+        encs = tok.encode_batch_fast(texts[i : i + STATS_ENCODE_BATCH], add_special_tokens=False)
+        for j, e in enumerate(encs):
+            lengths[i + j] = len(e.ids)
+            unk += e.ids.count(unk_id)
+    total_tokens = int(lengths.sum())
+    by_label: dict[str, float] = {}
+    if labels is not None:
+        lab = np.asarray(labels)
+        by_label = {str(v): round(float(lengths[lab == v].mean()), 1) for v in np.unique(lab)}
+    return SplitStats(
+        split=split,
+        file=path.name,
+        rows=len(texts),
+        mean_tokens=round(float(lengths.mean()), 1),
+        median_tokens=float(np.median(lengths)),
+        p95_tokens=float(np.percentile(lengths, 95)),
+        max_tokens=int(lengths.max()),
+        chars_per_token=round(sum(map(len, texts)) / max(total_tokens, 1), 3),
+        pct_truncated=round(100.0 * float((lengths + 2 > max_len).mean()), 2),
+        unk_rate=unk / max(total_tokens, 1),
+        mean_tokens_by_label=by_label,
+    )
+
+
+def example_encodings(tok: Tokenizer) -> list[dict]:
+    tok.no_truncation()
+    tok.no_padding()
+    out = []
+    for name, text in EXAMPLES:
+        enc = tok.encode(text)
+        decoded = tok.decode(enc.ids)
+        out.append({
+            "name": name,
+            "text": text,
+            "n_tokens_with_specials": len(enc.ids),
+            "tokens": enc.tokens,
+            "decoded": decoded,
+            "round_trip_exact": decoded == text,
+            "round_trip_matches_nfkc": decoded == normalizers.NFKC().normalize_str(text),
+        })
+    return out
+
+
+def stats(cfg: dict, paths: TokenizerPaths) -> dict:
+    max_len = int(cfg["model"]["max_seq_len"])
+    tok = load_tokenizer(paths.tokenizer_json)
+    rows: list[SplitStats] = []
+    for split in ("pretrain", "finetune_train", "finetune_val", "test_holdout"):
+        f = paths.data_dir / SPLIT_FILES[split]
+        if not f.exists():
+            LOG.warning("%s missing, skipped", f)
+            continue
+        sample = PRETRAIN_STATS_SAMPLE if split == "pretrain" else None
+        t0 = common.timer()
+        rows.append(split_stats(tok, split, f, max_len, sample, int(cfg["seed"])))
+        LOG.info("%s: %d rows in %.1fs", split, rows[-1].rows, common.timer() - t0)
+    examples = example_encodings(tok)
+
+    report = _read_report(paths)
+    report["stats"] = {
+        "max_seq_len": max_len,
+        "vocab_size": tok.get_vocab_size(),
+        "pretrain_sample_rows": PRETRAIN_STATS_SAMPLE,
+        "splits": [asdict(r) for r in rows],
+        "examples": examples,
+    }
+    common.write_json(report, paths.report_json)
+    _print_report(rows, examples, max_len, tok.get_vocab_size())
+    return report
+
+
+def _print_report(rows: list[SplitStats], examples: list[dict], max_len: int, vocab: int) -> None:
+    print(f"\n### Tokenizer statistics (vocab {vocab:,}, max_seq_len {max_len}; token counts exclude [CLS]/[SEP])\n")
+    print(f"| split | rows | mean tok | median | p95 | max | chars/tok | % > {max_len} | [UNK] rate | mean tok by label |")
+    print("|---|---:|---:|---:|---:|---:|---:|---:|---:|---|")
+    for r in rows:
+        lab = ", ".join(f"{k}: {v}" for k, v in r.mean_tokens_by_label.items()) or "-"
+        print(f"| {r.split} | {r.rows:,} | {r.mean_tokens:,.1f} | {r.median_tokens:,.0f} | {r.p95_tokens:,.0f} | "
+              f"{r.max_tokens:,} | {r.chars_per_token:.2f} | {r.pct_truncated:.1f}% | {r.unk_rate:.2e} | {lab} |")
+    print("\n### Example encodings\n")
+    for e in examples:
+        shown = " ".join(e["tokens"][:48]) + (" ..." if len(e["tokens"]) > 48 else "")
+        print(f"**{e['name']}** ({e['n_tokens_with_specials']} tokens; round trip exact={e['round_trip_exact']}, "
+              f"matches NFKC(input)={e['round_trip_matches_nfkc']})")
+        print(f"- tokens: `{shown}`")
+        print(f"- decoded: {e['decoded']!r}\n")
+
+
+# -------------------------------------------------------------------------------------------- CLI
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--config", default="config.yaml")
+    ap.add_argument("--data-dir", default=None, help=argparse.SUPPRESS)  # self-test override
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    tr = sub.add_parser("train", help="train byte-level BPE on pretrain + finetune_train")
+    tr.add_argument("--vocab-size", type=int, default=None)
+    tr.add_argument("--limit-docs", type=int, default=None, help="dev only: cap rows read per training file")
+    sub.add_parser("stats", help="token-length / truncation / [UNK] statistics per split")
+    args = ap.parse_args(argv)
+
+    cfg = common.load_config(args.config)
+    paths = resolve_paths(cfg, args.data_dir)
+    global LOG
+    LOG = common.setup_logging("tokenizer", paths.log_dir)
+    common.set_seed(int(cfg["seed"]))
+    if args.cmd == "train":
+        train(cfg, paths, args.vocab_size, args.limit_docs)
+    else:
+        stats(cfg, paths)
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())