# Release Checklist — Piko-9b 1.0.0

Run 2026-07-29. Every line is either **PASS** with evidence, **FAIL** with the reason, or
**NOT RUN** with why. Nothing is marked done on the strength of an assumption.

## Model integrity

| Check | Status | Evidence |
|---|---|---|
| `config.json` parses | **PASS** | `scripts/audit_repository.py` |
| Tokenizer loads | **PASS** | `AutoTokenizer` — `TokenizersBackend` |
| Processor loads | **PASS** | `Qwen3VLProcessor`, **requires `torchvision`** |
| Model loads without `trust_remote_code` | **PASS** | `AutoModelForMultimodalLM`, no `auto_map`, no `.py` in repo |
| Parameter count verified | **PASS** | 9,653,104,368 from safetensors headers |
| Shard integrity | **PASS** | 11 shards, 775 tensors, all BF16, 0 unexplained keys |
| Weights unchanged from previous release | **PASS** | Byte-identical; this release changes documentation only |

## Provenance

| Check | Status | Evidence |
|---|---|---|
| Base model identified | **PASS** | `deepreinforce-ai/Ornith-1.0-9B` + `Qwen/Qwen3.5-9B` |
| Lineage verified at tensor level | **PASS** | 427/427 language + 348/348 vision tensors matched bitwise |
| Adapter presence tested | **PASS** | Both probed adapters **not merged**; `\|cand−lang\| = 0` exactly |
| Ornith's own base model | **NOT VERIFIED** | Card declares none; structural identity to Qwen3.5-9B is an inference |
| Published checkpoint byte-compared to Hub | **NOT RUN** | Verified by file name and byte size only; 21 GB re-download not performed |

## Documentation honesty

| Check | Status | Evidence |
|---|---|---|
| No fabricated numbers | **PASS** | `scripts/validate_model_card.py` cross-checks every table percentage |
| Unrun benchmarks labelled | **PASS** | 9 standard benchmarks marked **Not run** |
| Base-model scores not presented as Piko's | **PASS** | Misattributed scores removed; explicit warning added |
| Unsupported capabilities not advertised | **PASS** | Video/audio marked unsupported; identity caveat stated |
| Failures recorded, not omitted | **PASS** | 5 suite failures adjudicated in the card; 32K OOM recorded |
| Model card metadata validates | **PASS** | 0 errors, 0 warnings |
| Relative links resolve | **PASS** | Checked by `validate_model_card.py` |

## Hygiene

| Check | Status | Evidence |
|---|---|---|
| No secrets | **PASS** | Token/key/private-key scan over all text files: 0 hits |
| No absolute local paths in shipped files | **PASS** | Result files normalised to repo ids; remaining mentions are deliberate documentation |
| `config.json` path leak fixed | **PASS (proposed)** | Correction specified in audit §8; **not yet applied to the Hub** |
| Regression test for path leak | **PASS** | `test_config_contains_no_absolute_paths` |
| `.gitignore` excludes weights | **PASS** | |

## Code quality

| Check | Status | Evidence |
|---|---|---|
| Syntax check, all Python | **PASS** | `compileall` clean |
| Lint | **PASS** | `ruff check` — all checks passed |
| Format | **PASS** | `ruff format --check` — 27 files formatted |
| Fast tests | **PASS** | **10 passed**, 14 deselected, no weights or GPU needed |
| Slow tests | **NOT RUN** | 14 collected; require `PIKO_MODEL_PATH`. Behaviour they assert was verified directly by `scripts/validate_inference.py` |

## Functional validation

| Check | Status | Evidence |
|---|---|---|
| Text generation | **PASS** | `reports/inference_validation.json` — 12/12 |
| Image / OCR | **PASS** | Read "NORTHGATE HARDWARE" and 22.75 correctly |
| Document JSON extraction | **PASS** | 10/10 on custom suite |
| Chart reading | **PASS** | Read all four bar values correctly |
| Multi-turn | **PASS** | |
| Batch inference | **PASS** | |
| Greedy determinism | **PASS** | Identical across repeats |
| Long context | **PASS** | Needle found at 14,429 tokens; suite passes at 2K/8K/32K |
| Streaming | **PASS (code)** | Implemented in `examples/inference_cli.py`; not separately measured |
| bfloat16 unquantized | **NOT RUN** | Needs ~22 GB; test GPU has 15.92 GB |
| float16 | **NOT RUN** | Supported by flag; not exercised |
| 8-bit | **NOT RUN** | |
| CPU loading | **NOT RUN** | ~38.6 GB float32 and no fast path; not viable |
| `device_map="auto"` | **FAIL — by design** | Produces `!!!!!` degenerate output; documented as unsupported |
| Video input | **NOT RUN** | No fixture built |

## Evaluation

| Check | Status | Evidence |
|---|---|---|
| Smoke evaluation | **PASS** | **8/8 required** |
| Custom suite, Piko-9b | **PASS** | **65/70** |
| Custom suite, base model | **PASS** | **63/70**, identical settings |
| Like-for-like comparison | **PASS** | Settings matched; tool would have refused otherwise |
| Statistical honesty | **PASS** | Wilson intervals; every category reads *No significant difference* |
| Standard public benchmarks | **NOT RUN** | 1–3 h per model each, ×2 for baseline; scripts provided |

## Performance

| Check | Status | Evidence |
|---|---|---|
| Cold load | **PASS** | 101–119 s NVMe |
| Peak VRAM | **PASS** | 7.37 GB resident, 13.4 GB at 8K×4 |
| Host RAM | **PASS** | 1.28 GB RSS |
| Throughput, 3 contexts × 3 batches | **PASS** | 9/9 configurations |
| Image preprocessing | **PASS** | 13 ms median |
| 32,768-token single forward | **FAIL — recorded** | OOM on 15.92 GB; 32K retrieval via `generate()` works |

## Quantization

| Check | Status | Evidence |
|---|---|---|
| 4-bit NF4 | **PASS** | All results measured this way; vision path intact |
| 8-bit | **NOT RUN** | |
| GGUF / AWQ / GPTQ | **NOT PRODUCED** | Scripts provided, unexecuted, documented as unvalidated |
| Quantized artefact published | **NO** | Nothing to validate, nothing claimed |

## Blocking items before publishing

1. **Apply the `config.json` correction on the Hub** — remove the two absolute paths from
   `piko_composition` and set `transformers_version` to `5.5.0`. This is the only change that
   touches the model repository itself, and it is live right now.
2. **Decide what the repository is for.** The evidence says Piko-9b performs the same as
   `Qwen/Qwen3.5-9B` within measurement error. That is publishable, but only with the honesty this
   release adopts. Do not restore the old benchmark table.

## Not blocking, worth doing

* Run one standard benchmark pair (GSM8K 200 items is the cheapest useful one).
* Test the six unmerged Piko adapters against the composition — they may improve it, and nobody
  has checked.
* Build a video fixture and either support or explicitly disable the video path.
* Test image inputs outside rendered documents.