TRAINING_REPORT.md

# CodePit PlanGuard 0.2 Seed LoRA Report

## What We Did

We trained a real seed LoRA adapter for CodePit PlanGuard on Apple Silicon using
MLX-LM and the public `CodePit/OnchainPlanBench-Seed` dataset.

This is a proof-of-work artifact for the CodePit model loop:

dataset -> local training -> validation -> public adapter -> future agent competition

## Model

- Base model: `mlx-community/Qwen2.5-0.5B-Instruct-bf16`
- Adapter: `CodePit/PlanGuard-0.2-Seed-LoRA`
- Dataset: `CodePit/OnchainPlanBench-Seed`
- Training method: LoRA with prompt masking

## Local Training Result

- Masked test loss: `0.015`
- Masked test perplexity: `1.015`
- Validation rows: `10`

## Generation Evaluation

The same validation prompts were generated with the base model and with the
PlanGuard seed adapter. Outputs were scored with the public lightweight
OnchainPlanBench evaluator.

| Metric | Base model | PlanGuard seed LoRA |
|---|---:|---:|
| JSON parse rate | 0.000 | 1.000 |
| Verdict match | 0.000 | 0.800 |
| Required tools present | 0.000 | 0.900 |
| Forbidden tools avoided | 0.000 | 0.900 |
| Privacy mode match | 0.000 | 1.000 |
| Confirmation gates | 0.000 | 0.800 |

## What We Learned

- The local machine can train a small PlanGuard adapter without a GPU server.
- A tiny LoRA adapter can memorize and emit the strict JSON structure on the
  seed tasks.
- The current seed split is too small and synthetic to claim production safety.
- The next useful work is not more hype; it is expanding the benchmark with
  harder held-out cases and letting CodePit agents compete on measurable gains.

## Claim Boundary

This adapter is not a production wallet-safety model. It does not authorize
transactions, provide legal/compliance advice, or replace transaction simulation.
A future PlanGuard version should only be called improved after CodePit's
verifier scores it on held-out benchmark tasks.