Audit record for the Meganeura case study

Date: 2026-08-01 (America/Los_Angeles)

Decision

The original REPORT.md is not admissible as paper evidence. It is a useful engineering narrative, but several headline results were either evaluated on training examples, measured with a biased timing protocol, or produced by an incorrect DINOv3 graph. The case study can become paper material after the fresh Quest correctness and measurement gates in experiments/README.md are complete. Corrected three-seed held-out decoder evaluation and publication of the host artifact are now complete.

The defensible case-study claim is deliberately narrow:

A decoder graph is trained through Meganeura autodiff and Adam on the host, then its forward path is joined to a frozen DINOv3 encoder and compiled by the same Meganeura graph compiler/runtime for Vulkan inference in an Android XR application sharing Blade's graphics context and queue.

Training caches frozen encoder features and executes a batched decoder-only graph. Deployment executes a batch-one joined encoder/decoder graph. They share decoder construction, parameters, IR operations, compiler, and runtime; they are not literally the same complete graph.

Findings that invalidate historical numbers

Historical evidence	Audit finding	Disposition
Decoder PSNR in `REPORT.md`	The trainer reconstructed the first six cached training images and averaged their PSNR. There was no held-out split, per-image record, or seed replication.	Replaced by an upstream-defined Imagenette validation split, all 3,925 images, per-image PSNR/SSIM/MAE, exact hashes, and three initializations.
Isolated device timings	The benchmark used one untimed warmup and reported the minimum retained sample as throughput. Clock ramp and process-to-process state were not controlled.	Rerun with at least five warmups, 20 retained samples, median/IQR/raw samples, and three fresh processes in a declared headset state.
DINOv3 encoder correctness	The app passed `[tokens, hidden]` data to Meganeura's NCHW-flat `mul_per_channel` as if it broadcast over the trailing dimension. Only the first token received the intended LayerScale update; later token residual branches were effectively suppressed.	Fixed by transposing to `[hidden, tokens]`, flattening for the NCHW operator, applying the hidden gain with `spatial=tokens`, reshaping, and transposing back. All old decoders and device timings are invalid.
Live XR “inference Hz”	Two asynchronous eye completions were combined into one count, and the raw-camera toggle still submitted hidden inference.	App records each eye independently and true raw-camera mode bypasses the model.
240-pixel, f16, and operation-share conclusions	These were microbenchmark observations or extrapolations, not matched end-to-end experiments or a deployed-graph profile.	Exclude causal or “closed” claims unless the matched experiments are run.

The invalid pre-fix decoder runs are preserved locally under C:\tmp\dinovision-runs; they must not be uploaded or cited. The corrected matrix uses a separate root so no artifact can be mistaken for a valid run.

Independent encoder validation after the fix

The reference is facebook/dinov3-vits16-pretrain-lvd1689m, executed in Torch 2.13.0+cpu and Transformers 5.14.1 from the same deterministic normalized pixel tensor and exact checkpoint. Meganeura uses f32 scalar matmul on the RTX 3050 for this comparison. Thresholds were declared before the corrected run: relative L2 at most 0.01, CLS cosine above 0.999, and every patch-token cosine above 0.999.

Depth	Relative L2	CLS cosine	Worst patch cosine	Result
1 layer	0.000781	1.000000	0.999990	pass
3 layers (deployed)	0.001403	1.000000	0.999995	pass
12 layers (upstream depth)	0.002253	0.999997	0.999996	pass

At the first layer, embeddings, LayerNorm, Q/K/V projections, and RoPE agree to the displayed six decimal places. Attention differs by 0.005852 relative L2; after learned LayerScale the residual differs by 0.000566. Exact-shape tests independently compare Meganeura attention and the corrected trailing LayerScale broadcast against CPU implementations.

A deliberately small deterministic replay also ran the same 20-update, 16-image, seed-20260801 cell twice. The two serialized decoders are byte-identical (SHA-256 2959b06784576bb1bc495b4e0a0d2f4aeebb216d56f7fc078843df4b7ec12686). This validates the seeded code path and serialization, not full-run repeatability across machines or drivers.

The audit also recommends hardening Meganeura's mul_per_channel builder. Its documented flat NCHW input/gate contract is not currently asserted, and the compiler derives the dispatch length from shape[0]. Validating rank, element count, channel/spatial divisibility, and gate length at graph construction would have rejected the original misuse before execution. This is not needed to reinterpret the corrected results, but it is a worthwhile paper-artifact change in Meganeura itself.

Evidence status

Complete: the public mad-bot/dinovision artifact binds all three decoders to the corrected source snapshots, DINO checkpoint hash, dataset-manifest hash, depth, resolution, and seeds, and retains the fail-closed validation manifest.
Complete: fixed-frame host/Quest comparison passes for bit-identical patches, encoder output (relative L2 0.001059, cosine 0.99999944, minimum token cosine 0.999973), spatial decoder input (relative L2 0.001063), and reconstruction (relative L2 0.000450, cosine 0.99999994).
Fresh isolated Quest timings and live-worn XR chunk sweep from the corrected graph, with raw samples and state snapshots.
Clean, immutable DinoVision and Meganeura revisions. The current worktree is a development freeze candidate, not yet a citable revision.
A validation-clean debug host context. Numerical tests pass, but the resolved Blade Vulkan setup currently emits two device-creation VUIDs; this is distinct from the corrected Meganeura tensor results and should be tracked rather than hidden.

Until those items are complete, the correct paper status is promising but not merge-ready.